yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-07-06 16:49:03 +00:00
Code Issues Packages Projects Releases Wiki Activity
datahub/metadata-ingestion/docs/sources/dremio/dremio_pre.md
sagar-salvi-apptware 5d17ecb290
feat(ingest/dremio): Dremio Source Ingestion (#11598) 
Co-authored-by: Jonny Dixon <jonny.dixon@acryl.io>
Co-authored-by: Jonny Dixon <45681293+acrylJonny@users.noreply.github.com>
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2024-11-05 22:06:35 +05:30

1.0 KiB
Raw Permalink Blame History

Setup

This integration pulls metadata directly from the Dremio APIs.

You'll need to have a Dremio instance up and running with access to the necessary datasets, and API access should be enabled with a valid token.

The API token should have the necessary permissions to read metadata and retrieve lineage.

Steps to Get the Required Information

  1. Generate an API Token:

    • Log in to your Dremio instance.
    • Navigate to your user profile in the top-right corner.
    • Select Generate API Token to create an API token for programmatic access.

  2. Permissions:

    • The token should have read-only or admin permissions that allow it to:
      • View all datasets (physical and virtual).
      • Access all spaces, folders, and sources.
      • Retrieve dataset and column-level lineage information.

  3. Verify External Data Source Permissions:

    • If Dremio is connected to external data sources (e.g., AWS S3, relational databases), ensure that Dremio has access to the credentials required for querying those sources.