yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-08-20 23:28:01 +00:00
Code Issues Packages Projects Releases Wiki Activity
datahub/metadata-ingestion/docs/sources/dremio/dremio.md
sagar-salvi-apptware 5d17ecb290
feat(ingest/dremio): Dremio Source Ingestion (#11598) 
Co-authored-by: Jonny Dixon <jonny.dixon@acryl.io>
Co-authored-by: Jonny Dixon <45681293+acrylJonny@users.noreply.github.com>
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2024-11-05 22:06:35 +05:30

793 B
Raw Blame History

Starter Receipe for Dremio Cloud Instance

source:
  type: dremio
  config:
    # Authentication details
    authentication_method: PAT        # Use Personal Access Token for authentication
    password: <your_api_token>        # Replace <your_api_token> with your Dremio Cloud API token
    is_dremio_cloud: True             # Set to True for Dremio Cloud instances
    dremio_cloud_project_id: <project_id>  # Provide the Project ID for Dremio Cloud

    # Enable query lineage tracking
    include_query_lineage: True       

    #Optional
    source_mappings:
      - platform: s3                 
        source_name: samples       
    
    # Optional
    schema_pattern:
      allow:
        - "<source_name>.<table_name>" 

sink:
    # Define your sink configuration here