datahub/metadata-ingestion/docs/sources/sigma/sigma_pre.md

Integration Details

This source extracts the following:

• Workspaces and workbooks within that workspaces as Container.
• Sigma Datasets as Datahub Datasets.
• Pages as Datahub dashboards and elements present inside pages as charts.

Configuration Notes

1. Refer doc to generate an API client credentials.
2. Provide the generated Client ID and Secret in Recipe.

We have observed issues with the Sigma API, where certain API endpoints do not return the expected results, even when the user is an admin. In those cases, a workaround is to manually add the user associated with the Client ID/Secret to each workspace with missing metadata. Empty workspaces are listed in the ingestion report in the logs with the key empty_workspaces.

Concept mapping

Sigma	Datahub	Notes
Workspace	Container	SubType "Sigma Workspace"
Workbook	Dashboard	SubType "Sigma Workbook"
Page	Dashboard
Element	Chart
Dataset	Dataset	SubType "Sigma Dataset"
User	User (a.k.a CorpUser)	Optionally Extracted

Advanced Configurations

Chart source platform mapping

If you want to provide platform details(platform name, platform instance and env) for chart's all external upstream data sources, then you can use chart_sources_platform_mapping as below:

Example - For just one specific chart's external upstream data sources

chart_sources_platform_mapping:
  "workspace_name/workbook_name/chart_name_1":
    data_source_platform: snowflake
    platform_instance: new_instance
    env: PROD

  "workspace_name/folder_name/workbook_name/chart_name_2":
    data_source_platform: postgres
    platform_instance: cloud_instance
    env: DEV

Example - For all charts within one specific workbook

chart_sources_platform_mapping:
  "workspace_name/workbook_name_1":
    data_source_platform: snowflake
    platform_instance: new_instance
    env: PROD

  "workspace_name/folder_name/workbook_name_2":
    data_source_platform: snowflake
    platform_instance: new_instance
    env: PROD

Example - For all workbooks charts within one specific workspace

chart_sources_platform_mapping:
  "workspace_name":
    data_source_platform: snowflake
    platform_instance: new_instance
    env: PROD

Example - All workbooks use the same connection

chart_sources_platform_mapping:
  "*":
    data_source_platform: snowflake
    platform_instance: new_instance
    env: PROD