datahub/metadata-ingestion/docs/sources/hex

This connector ingests Hex assets into DataHub.

Concept Mapping

Hex Concept	DataHub Concept	Notes
"hex"	Data Platform
Workspace	Container
Project	Dashboard	Subtype Project
Component	Dashboard	Subtype Component
Collection	Tag

Other Hex concepts are not mapped to DataHub entities yet.

Limitations

Currently, the Hex API has some limitations that affect the completeness of the extracted metadata:

1. Projects and Components Relationship: The API does not support fetching the many-to-many relationship between Projects and their Components.

2. Metadata Access: There is no direct method to retrieve metadata for Collections, Status, or Categories. This information is only available indirectly through references within Projects and Components.

Please keep these limitations in mind when working with the Hex connector.

For the Dataset - Hex Project lineage, the connector relies on the Hex query metadata feature. Therefore, in order to extract lineage information, the required setup must include:

• A separated warehouse ingestor (eg BigQuery, Snowflake, Redshift, ...) with use_queries_v2 enabled in order to fetch Queries. This will ingest the queries into DataHub as Query entities and the ones triggered by Hex will include the corresponding Hex query metadata.
• A DataHub server with version >= SaaS 0.3.10 or > OSS 1.0.0 so the Query entities are properly indexed by source (Hex in this case) and so fetched and processed by the Hex ingestor in order to emit the Dataset - Project lineage.

Please note:

• Lineage is only captured for scheduled executions of the Project.
• In cases where queries are handled by hextoolkit, Hex query metadata is not injected, which prevents capturing lineage.