diff --git a/openmetadata-docs/content/v1.6.x/connectors/database/databricks/index.md b/openmetadata-docs/content/v1.6.x/connectors/database/databricks/index.md index d9b517bc025..4041080f2b6 100644 --- a/openmetadata-docs/content/v1.6.x/connectors/database/databricks/index.md +++ b/openmetadata-docs/content/v1.6.x/connectors/database/databricks/index.md @@ -20,6 +20,7 @@ In this section, we provide guides and references to use the Databricks connecto Configure and schedule Databricks metadata and profiler workflows from the OpenMetadata UI: +- [Requirements](#requirements) - [Unity Catalog](#unity-catalog) - [Metadata Ingestion](#metadata-ingestion) - [Query Usage](/connectors/ingestion/workflows/usage) @@ -33,6 +34,51 @@ Configure and schedule Databricks metadata and profiler workflows from the OpenM {% partial file="/v1.6/connectors/external-ingestion-deployment.md" /%} +## Requirements + +### Python Requirements + +{% partial file="/v1.8/connectors/python-requirements.md" /%} + +To run the Databricks ingestion, you will need to install: + +```bash +pip3 install "openmetadata-ingestion[databricks]" +``` + +### Permission Requirement + +To enable full functionality of metadata extraction, profiling, usage, and lineage features in OpenMetadata, the following permissions must be granted to the relevant users in your Databricks environment. + +### Metadata and Profiling Permissions + +These permissions are required on the catalogs, schemas, and tables from which metadata and profiling information will be ingested. + +```sql +GRANT USE CATALOG ON CATALOG TO ``; +GRANT USE SCHEMA ON SCHEMA TO ``; +GRANT SELECT ON TABLE TO ``; +``` + +Ensure these grants are applied to all relevant tables for metadata ingestion and profiling operations. + +### Usage and Lineage + +These permissions enable OpenMetadata to extract query history and construct lineage information. + +```sql +GRANT SELECT ON SYSTEM.QUERY.HISTORY TO ``; +GRANT USE SCHEMA ON SCHEMA system.query TO ``; +``` + +These permissions allow access to Databricks system tables that track query activity, enabling lineage and usage statistics generation. + +{% note %} + +Adjust , , , and according to your specific deployment and security requirements. + +{% /note %} + ## Unity Catalog If you are using unity catalog in Databricks, then checkout the [Unity Catalog](/connectors/database/unity-catalog) connector. diff --git a/openmetadata-docs/content/v1.6.x/connectors/database/databricks/yaml.md b/openmetadata-docs/content/v1.6.x/connectors/database/databricks/yaml.md index eb969dcd3b3..ead7a63b4c1 100644 --- a/openmetadata-docs/content/v1.6.x/connectors/database/databricks/yaml.md +++ b/openmetadata-docs/content/v1.6.x/connectors/database/databricks/yaml.md @@ -41,6 +41,39 @@ To run the Databricks ingestion, you will need to install: pip3 install "openmetadata-ingestion[databricks]" ``` +### Permission Requirement + +To enable full functionality of metadata extraction, profiling, usage, and lineage features in OpenMetadata, the following permissions must be granted to the relevant users in your Databricks environment. + +### Metadata and Profiling Permissions + +These permissions are required on the catalogs, schemas, and tables from which metadata and profiling information will be ingested. + +```sql +GRANT USE CATALOG ON CATALOG TO ``; +GRANT USE SCHEMA ON SCHEMA TO ``; +GRANT SELECT ON TABLE TO ``; +``` + +Ensure these grants are applied to all relevant tables for metadata ingestion and profiling operations. + +### Usage and Lineage + +These permissions enable OpenMetadata to extract query history and construct lineage information. + +```sql +GRANT SELECT ON SYSTEM.QUERY.HISTORY TO ``; +GRANT USE SCHEMA ON SCHEMA system.query TO ``; +``` + +These permissions allow access to Databricks system tables that track query activity, enabling lineage and usage statistics generation. + +{% note %} + +Adjust , , , and according to your specific deployment and security requirements. + +{% /note %} + ## Metadata Ingestion All connectors are defined as JSON Schemas. diff --git a/openmetadata-docs/content/v1.6.x/connectors/database/unity-catalog/index.md b/openmetadata-docs/content/v1.6.x/connectors/database/unity-catalog/index.md index 2b8decb81be..135791c222f 100644 --- a/openmetadata-docs/content/v1.6.x/connectors/database/unity-catalog/index.md +++ b/openmetadata-docs/content/v1.6.x/connectors/database/unity-catalog/index.md @@ -16,6 +16,7 @@ In this section, we provide guides and references to use the Unity Catalog conne Configure and schedule Unity Catalog metadata workflow from the OpenMetadata UI: +- [Requirements](#requirements) - [Metadata Ingestion](#metadata-ingestion) - [Query Usage](/connectors/ingestion/workflows/usage) - [Data Quality](/how-to-guides/data-quality-observability/quality) @@ -27,6 +28,50 @@ Configure and schedule Unity Catalog metadata workflow from the OpenMetadata UI: {% partial file="/v1.6/connectors/external-ingestion-deployment.md" /%} +## Requirements + +### Python Requirements + +{% partial file="/v1.8/connectors/python-requirements.md" /%} + +To run the Unity Catalog ingestion, you will need to install: + +```bash +pip3 install "openmetadata-ingestion[databricks]" +``` +### Permission Requirement + +To enable full functionality of metadata extraction, profiling, usage, and lineage features in OpenMetadata, the following permissions must be granted to the relevant users in your Databricks environment. + +### Metadata and Profiling Permissions + +These permissions are required on the catalogs, schemas, and tables from which metadata and profiling information will be ingested. + +```sql +GRANT USE CATALOG ON CATALOG TO ``; +GRANT USE SCHEMA ON SCHEMA TO ``; +GRANT SELECT ON TABLE TO ``; +``` + +Ensure these grants are applied to all relevant tables for metadata ingestion and profiling operations. + +### Usage and Lineage + +These permissions enable OpenMetadata to extract query history and construct lineage information. + +```sql +GRANT SELECT ON SYSTEM.QUERY.HISTORY TO ``; +GRANT USE SCHEMA ON SCHEMA system.query TO ``; +``` + +These permissions allow access to Databricks system tables that track query activity, enabling lineage and usage statistics generation. + +{% note %} + +Adjust , , , and according to your specific deployment and security requirements. + +{% /note %} + ## Metadata Ingestion {% partial diff --git a/openmetadata-docs/content/v1.6.x/connectors/database/unity-catalog/yaml.md b/openmetadata-docs/content/v1.6.x/connectors/database/unity-catalog/yaml.md index 1b0fcec47f5..ff5a0ce73dd 100644 --- a/openmetadata-docs/content/v1.6.x/connectors/database/unity-catalog/yaml.md +++ b/openmetadata-docs/content/v1.6.x/connectors/database/unity-catalog/yaml.md @@ -35,6 +35,39 @@ To run the Unity Catalog ingestion, you will need to install: pip3 install "openmetadata-ingestion[databricks]" ``` +### Permission Requirement + +To enable full functionality of metadata extraction, profiling, usage, and lineage features in OpenMetadata, the following permissions must be granted to the relevant users in your Databricks environment. + +### Metadata and Profiling Permissions + +These permissions are required on the catalogs, schemas, and tables from which metadata and profiling information will be ingested. + +```sql +GRANT USE CATALOG ON CATALOG TO ``; +GRANT USE SCHEMA ON SCHEMA TO ``; +GRANT SELECT ON TABLE TO ``; +``` + +Ensure these grants are applied to all relevant tables for metadata ingestion and profiling operations. + +### Usage and Lineage + +These permissions enable OpenMetadata to extract query history and construct lineage information. + +```sql +GRANT SELECT ON SYSTEM.QUERY.HISTORY TO ``; +GRANT USE SCHEMA ON SCHEMA system.query TO ``; +``` + +These permissions allow access to Databricks system tables that track query activity, enabling lineage and usage statistics generation. + +{% note %} + +Adjust , , , and according to your specific deployment and security requirements. + +{% /note %} + ## Metadata Ingestion All connectors are defined as JSON Schemas. diff --git a/openmetadata-docs/content/v1.7.x/connectors/database/databricks/index.md b/openmetadata-docs/content/v1.7.x/connectors/database/databricks/index.md index 2adc16ae3b7..4763726d7a0 100644 --- a/openmetadata-docs/content/v1.7.x/connectors/database/databricks/index.md +++ b/openmetadata-docs/content/v1.7.x/connectors/database/databricks/index.md @@ -20,6 +20,7 @@ In this section, we provide guides and references to use the Databricks connecto Configure and schedule Databricks metadata and profiler workflows from the OpenMetadata UI: +- [Requirements](#requirements) - [Unity Catalog](#unity-catalog) - [Metadata Ingestion](#metadata-ingestion) - [Query Usage](/connectors/ingestion/workflows/usage) @@ -36,6 +37,51 @@ Configure and schedule Databricks metadata and profiler workflows from the OpenM {% partial file="/v1.7/connectors/external-ingestion-deployment.md" /%} +## Requirements + +### Python Requirements + +{% partial file="/v1.8/connectors/python-requirements.md" /%} + +To run the Databricks ingestion, you will need to install: + +```bash +pip3 install "openmetadata-ingestion[databricks]" +``` + +### Permission Requirement + +To enable full functionality of metadata extraction, profiling, usage, and lineage features in OpenMetadata, the following permissions must be granted to the relevant users in your Databricks environment. + +### Metadata and Profiling Permissions + +These permissions are required on the catalogs, schemas, and tables from which metadata and profiling information will be ingested. + +```sql +GRANT USE CATALOG ON CATALOG TO ``; +GRANT USE SCHEMA ON SCHEMA TO ``; +GRANT SELECT ON TABLE TO ``; +``` + +Ensure these grants are applied to all relevant tables for metadata ingestion and profiling operations. + +### Usage and Lineage + +These permissions enable OpenMetadata to extract query history and construct lineage information. + +```sql +GRANT SELECT ON SYSTEM.QUERY.HISTORY TO ``; +GRANT USE SCHEMA ON SCHEMA system.query TO ``; +``` + +These permissions allow access to Databricks system tables that track query activity, enabling lineage and usage statistics generation. + +{% note %} + +Adjust , , , and according to your specific deployment and security requirements. + +{% /note %} + ## Unity Catalog If you are using unity catalog in Databricks, then checkout the [Unity Catalog](/connectors/database/unity-catalog) connector. diff --git a/openmetadata-docs/content/v1.7.x/connectors/database/databricks/yaml.md b/openmetadata-docs/content/v1.7.x/connectors/database/databricks/yaml.md index dc1eff1a73b..6893c7115a3 100644 --- a/openmetadata-docs/content/v1.7.x/connectors/database/databricks/yaml.md +++ b/openmetadata-docs/content/v1.7.x/connectors/database/databricks/yaml.md @@ -43,6 +43,39 @@ To run the Databricks ingestion, you will need to install: pip3 install "openmetadata-ingestion[databricks]" ``` +### Permission Requirement + +To enable full functionality of metadata extraction, profiling, usage, and lineage features in OpenMetadata, the following permissions must be granted to the relevant users in your Databricks environment. + +### Metadata and Profiling Permissions + +These permissions are required on the catalogs, schemas, and tables from which metadata and profiling information will be ingested. + +```sql +GRANT USE CATALOG ON CATALOG TO ``; +GRANT USE SCHEMA ON SCHEMA TO ``; +GRANT SELECT ON TABLE TO ``; +``` + +Ensure these grants are applied to all relevant tables for metadata ingestion and profiling operations. + +### Usage and Lineage + +These permissions enable OpenMetadata to extract query history and construct lineage information. + +```sql +GRANT SELECT ON SYSTEM.QUERY.HISTORY TO ``; +GRANT USE SCHEMA ON SCHEMA system.query TO ``; +``` + +These permissions allow access to Databricks system tables that track query activity, enabling lineage and usage statistics generation. + +{% note %} + +Adjust , , , and according to your specific deployment and security requirements. + +{% /note %} + ## Metadata Ingestion All connectors are defined as JSON Schemas. diff --git a/openmetadata-docs/content/v1.7.x/connectors/database/unity-catalog/index.md b/openmetadata-docs/content/v1.7.x/connectors/database/unity-catalog/index.md index 4806de3220e..9534e1d8961 100644 --- a/openmetadata-docs/content/v1.7.x/connectors/database/unity-catalog/index.md +++ b/openmetadata-docs/content/v1.7.x/connectors/database/unity-catalog/index.md @@ -16,6 +16,7 @@ In this section, we provide guides and references to use the Unity Catalog conne Configure and schedule Unity Catalog metadata workflow from the OpenMetadata UI: +- [Requirements](#requirements) - [Metadata Ingestion](#metadata-ingestion) - [Data Quality](/how-to-guides/data-quality-observability/quality) - [Lineage](/connectors/ingestion/lineage) @@ -29,6 +30,50 @@ Configure and schedule Unity Catalog metadata workflow from the OpenMetadata UI: {% partial file="/v1.7/connectors/external-ingestion-deployment.md" /%} +## Requirements + +### Python Requirements + +{% partial file="/v1.8/connectors/python-requirements.md" /%} + +To run the Unity Catalog ingestion, you will need to install: + +```bash +pip3 install "openmetadata-ingestion[databricks]" +``` +### Permission Requirement + +To enable full functionality of metadata extraction, profiling, usage, and lineage features in OpenMetadata, the following permissions must be granted to the relevant users in your Databricks environment. + +### Metadata and Profiling Permissions + +These permissions are required on the catalogs, schemas, and tables from which metadata and profiling information will be ingested. + +```sql +GRANT USE CATALOG ON CATALOG TO ``; +GRANT USE SCHEMA ON SCHEMA TO ``; +GRANT SELECT ON TABLE TO ``; +``` + +Ensure these grants are applied to all relevant tables for metadata ingestion and profiling operations. + +### Usage and Lineage + +These permissions enable OpenMetadata to extract query history and construct lineage information. + +```sql +GRANT SELECT ON SYSTEM.QUERY.HISTORY TO ``; +GRANT USE SCHEMA ON SCHEMA system.query TO ``; +``` + +These permissions allow access to Databricks system tables that track query activity, enabling lineage and usage statistics generation. + +{% note %} + +Adjust , , , and according to your specific deployment and security requirements. + +{% /note %} + ## Metadata Ingestion {% partial diff --git a/openmetadata-docs/content/v1.7.x/connectors/database/unity-catalog/yaml.md b/openmetadata-docs/content/v1.7.x/connectors/database/unity-catalog/yaml.md index 0c0e5f27198..ad476ce04b8 100644 --- a/openmetadata-docs/content/v1.7.x/connectors/database/unity-catalog/yaml.md +++ b/openmetadata-docs/content/v1.7.x/connectors/database/unity-catalog/yaml.md @@ -38,6 +38,39 @@ To run the Unity Catalog ingestion, you will need to install: pip3 install "openmetadata-ingestion[databricks]" ``` +### Permission Requirement + +To enable full functionality of metadata extraction, profiling, usage, and lineage features in OpenMetadata, the following permissions must be granted to the relevant users in your Databricks environment. + +### Metadata and Profiling Permissions + +These permissions are required on the catalogs, schemas, and tables from which metadata and profiling information will be ingested. + +```sql +GRANT USE CATALOG ON CATALOG TO ``; +GRANT USE SCHEMA ON SCHEMA TO ``; +GRANT SELECT ON TABLE TO ``; +``` + +Ensure these grants are applied to all relevant tables for metadata ingestion and profiling operations. + +### Usage and Lineage + +These permissions enable OpenMetadata to extract query history and construct lineage information. + +```sql +GRANT SELECT ON SYSTEM.QUERY.HISTORY TO ``; +GRANT USE SCHEMA ON SCHEMA system.query TO ``; +``` + +These permissions allow access to Databricks system tables that track query activity, enabling lineage and usage statistics generation. + +{% note %} + +Adjust , , , and according to your specific deployment and security requirements. + +{% /note %} + ## Metadata Ingestion All connectors are defined as JSON Schemas. diff --git a/openmetadata-docs/content/v1.8.x-SNAPSHOT/connectors/database/databricks/index.md b/openmetadata-docs/content/v1.8.x-SNAPSHOT/connectors/database/databricks/index.md index 183cff1f434..7c5b13c60ab 100644 --- a/openmetadata-docs/content/v1.8.x-SNAPSHOT/connectors/database/databricks/index.md +++ b/openmetadata-docs/content/v1.8.x-SNAPSHOT/connectors/database/databricks/index.md @@ -20,6 +20,7 @@ In this section, we provide guides and references to use the Databricks connecto Configure and schedule Databricks metadata and profiler workflows from the OpenMetadata UI: +- [Requirements](#requirements) - [Unity Catalog](#unity-catalog) - [Metadata Ingestion](#metadata-ingestion) - [Query Usage](/connectors/ingestion/workflows/usage) @@ -36,6 +37,51 @@ Configure and schedule Databricks metadata and profiler workflows from the OpenM {% partial file="/v1.8/connectors/external-ingestion-deployment.md" /%} +## Requirements + +### Python Requirements + +{% partial file="/v1.8/connectors/python-requirements.md" /%} + +To run the Databricks ingestion, you will need to install: + +```bash +pip3 install "openmetadata-ingestion[databricks]" +``` + +### Permission Requirement + +To enable full functionality of metadata extraction, profiling, usage, and lineage features in OpenMetadata, the following permissions must be granted to the relevant users in your Databricks environment. + +### Metadata and Profiling Permissions + +These permissions are required on the catalogs, schemas, and tables from which metadata and profiling information will be ingested. + +```sql +GRANT USE CATALOG ON CATALOG TO ``; +GRANT USE SCHEMA ON SCHEMA TO ``; +GRANT SELECT ON TABLE TO ``; +``` + +Ensure these grants are applied to all relevant tables for metadata ingestion and profiling operations. + +### Usage and Lineage + +These permissions enable OpenMetadata to extract query history and construct lineage information. + +```sql +GRANT SELECT ON SYSTEM.QUERY.HISTORY TO ``; +GRANT USE SCHEMA ON SCHEMA system.query TO ``; +``` + +These permissions allow access to Databricks system tables that track query activity, enabling lineage and usage statistics generation. + +{% note %} + +Adjust , , , and according to your specific deployment and security requirements. + +{% /note %} + ## Unity Catalog If you are using unity catalog in Databricks, then checkout the [Unity Catalog](/connectors/database/unity-catalog) connector. diff --git a/openmetadata-docs/content/v1.8.x-SNAPSHOT/connectors/database/databricks/yaml.md b/openmetadata-docs/content/v1.8.x-SNAPSHOT/connectors/database/databricks/yaml.md index 64f60640dbf..695e9d3f12e 100644 --- a/openmetadata-docs/content/v1.8.x-SNAPSHOT/connectors/database/databricks/yaml.md +++ b/openmetadata-docs/content/v1.8.x-SNAPSHOT/connectors/database/databricks/yaml.md @@ -43,6 +43,39 @@ To run the Databricks ingestion, you will need to install: pip3 install "openmetadata-ingestion[databricks]" ``` +### Permission Requirement + +To enable full functionality of metadata extraction, profiling, usage, and lineage features in OpenMetadata, the following permissions must be granted to the relevant users in your Databricks environment. + +### Metadata and Profiling Permissions + +These permissions are required on the catalogs, schemas, and tables from which metadata and profiling information will be ingested. + +```sql +GRANT USE CATALOG ON CATALOG TO ``; +GRANT USE SCHEMA ON SCHEMA TO ``; +GRANT SELECT ON TABLE TO ``; +``` + +Ensure these grants are applied to all relevant tables for metadata ingestion and profiling operations. + +### Usage and Lineage + +These permissions enable OpenMetadata to extract query history and construct lineage information. + +```sql +GRANT SELECT ON SYSTEM.QUERY.HISTORY TO ``; +GRANT USE SCHEMA ON SCHEMA system.query TO ``; +``` + +These permissions allow access to Databricks system tables that track query activity, enabling lineage and usage statistics generation. + +{% note %} + +Adjust , , , and according to your specific deployment and security requirements. + +{% /note %} + ## Metadata Ingestion All connectors are defined as JSON Schemas. diff --git a/openmetadata-docs/content/v1.8.x-SNAPSHOT/connectors/database/unity-catalog/index.md b/openmetadata-docs/content/v1.8.x-SNAPSHOT/connectors/database/unity-catalog/index.md index a7c558005d0..6369b6cdada 100644 --- a/openmetadata-docs/content/v1.8.x-SNAPSHOT/connectors/database/unity-catalog/index.md +++ b/openmetadata-docs/content/v1.8.x-SNAPSHOT/connectors/database/unity-catalog/index.md @@ -16,6 +16,7 @@ In this section, we provide guides and references to use the Unity Catalog conne Configure and schedule Unity Catalog metadata workflow from the OpenMetadata UI: +- [Requirements](#requirements) - [Metadata Ingestion](#metadata-ingestion) - [Data Quality](/how-to-guides/data-quality-observability/quality) - [Lineage](/connectors/ingestion/lineage) @@ -29,6 +30,50 @@ Configure and schedule Unity Catalog metadata workflow from the OpenMetadata UI: {% partial file="/v1.8/connectors/external-ingestion-deployment.md" /%} +## Requirements + +### Python Requirements + +{% partial file="/v1.8/connectors/python-requirements.md" /%} + +To run the Unity Catalog ingestion, you will need to install: + +```bash +pip3 install "openmetadata-ingestion[databricks]" +``` +### Permission Requirement + +To enable full functionality of metadata extraction, profiling, usage, and lineage features in OpenMetadata, the following permissions must be granted to the relevant users in your Databricks environment. + +### Metadata and Profiling Permissions + +These permissions are required on the catalogs, schemas, and tables from which metadata and profiling information will be ingested. + +```sql +GRANT USE CATALOG ON CATALOG TO ``; +GRANT USE SCHEMA ON SCHEMA TO ``; +GRANT SELECT ON TABLE TO ``; +``` + +Ensure these grants are applied to all relevant tables for metadata ingestion and profiling operations. + +### Usage and Lineage + +These permissions enable OpenMetadata to extract query history and construct lineage information. + +```sql +GRANT SELECT ON SYSTEM.QUERY.HISTORY TO ``; +GRANT USE SCHEMA ON SCHEMA system.query TO ``; +``` + +These permissions allow access to Databricks system tables that track query activity, enabling lineage and usage statistics generation. + +{% note %} + +Adjust , , , and according to your specific deployment and security requirements. + +{% /note %} + ## Metadata Ingestion {% partial diff --git a/openmetadata-docs/content/v1.8.x-SNAPSHOT/connectors/database/unity-catalog/yaml.md b/openmetadata-docs/content/v1.8.x-SNAPSHOT/connectors/database/unity-catalog/yaml.md index f3ab1e70cad..e36da0f03a2 100644 --- a/openmetadata-docs/content/v1.8.x-SNAPSHOT/connectors/database/unity-catalog/yaml.md +++ b/openmetadata-docs/content/v1.8.x-SNAPSHOT/connectors/database/unity-catalog/yaml.md @@ -37,6 +37,38 @@ To run the Unity Catalog ingestion, you will need to install: ```bash pip3 install "openmetadata-ingestion[databricks]" ``` +### Permission Requirement + +To enable full functionality of metadata extraction, profiling, usage, and lineage features in OpenMetadata, the following permissions must be granted to the relevant users in your Databricks environment. + +### Metadata and Profiling Permissions + +These permissions are required on the catalogs, schemas, and tables from which metadata and profiling information will be ingested. + +```sql +GRANT USE CATALOG ON CATALOG TO ``; +GRANT USE SCHEMA ON SCHEMA TO ``; +GRANT SELECT ON TABLE TO ``; +``` + +Ensure these grants are applied to all relevant tables for metadata ingestion and profiling operations. + +### Usage and Lineage + +These permissions enable OpenMetadata to extract query history and construct lineage information. + +```sql +GRANT SELECT ON SYSTEM.QUERY.HISTORY TO ``; +GRANT USE SCHEMA ON SCHEMA system.query TO ``; +``` + +These permissions allow access to Databricks system tables that track query activity, enabling lineage and usage statistics generation. + +{% note %} + +Adjust , , , and according to your specific deployment and security requirements. + +{% /note %} ## Metadata Ingestion