yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-08-22 08:08:01 +00:00
Code Issues Packages Projects Releases Wiki Activity
datahub/metadata-ingestion/docs/sources/sigma/sigma_pre.md
Shubham Jagtap 90c1249e7d
feat(ingest/sigma): Sigma connector integration (#10037) 
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-04-15 20:18:31 -07:00

2.9 KiB
Raw Blame History

Integration Details

This source extracts the following:

  • Workspaces and workbooks within that workspaces as Container.
  • Sigma Datasets as Datahub Datasets.
  • Pages as Datahub dashboards and elements present inside pages as charts.

Configuration Notes

  1. Refer doc to generate an API client credentials.
  2. Provide the generated Client ID and Secret in Recipe.

Concept mapping

Sigma Datahub Notes
Workspace Container SubType "Sigma Workspace"
Workbook Container SubType "Sigma Workbook"
Page Dashboard
Element Chart
Dataset Dataset SubType "Sigma Dataset"
User User (a.k.a CorpUser) Optionally Extracted

Advanced Configurations

Chart source platform mapping

If you want to provide platform details(platform name, platform instance and env) for chart's all external upstream data sources, then you can use chart_sources_platform_mapping as below:

Example - For just one specific chart's external upstream data sources

    chart_sources_platform_mapping:
      'workspace_name/workbook_name/chart_name_1': 
        data_source_platform: snowflake
        platform_instance: new_instance
        env: PROD

      'workspace_name/folder_name/workbook_name/chart_name_2': 
        data_source_platform: postgres
        platform_instance: cloud_instance
        env: DEV

Example - For all charts within one specific workbook

    chart_sources_platform_mapping:
      'workspace_name/workbook_name_1': 
        data_source_platform: snowflake
        platform_instance: new_instance
        env: PROD
      
      'workspace_name/folder_name/workbook_name_2': 
        data_source_platform: snowflake
        platform_instance: new_instance
        env: PROD

Example - For all workbooks charts within one specific workspace

    chart_sources_platform_mapping:
      'workspace_name': 
        data_source_platform: snowflake
        platform_instance: new_instance
        env: PROD

Example - All workbooks use the same connection

    chart_sources_platform_mapping:
      '*': 
        data_source_platform: snowflake
        platform_instance: new_instance
        env: PROD