yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-06-27 05:03:31 +00:00
Code Issues Packages Projects Releases Wiki Activity
datahub/docs/quick-ingestion-guides/bigquery/overview.md
Tamas Nemeth 43f95d2b5c
Initial commit for bigquery ingestion guide (#6587) 
* Initial commit for bigquery ingestion guide

* Addressing PR review comments

* Fixing lint error

* Shorten titles

* Removing images

* update copy on overview.md

* update to setup steps with additional roles

* update configuration.md

* lowcasing overview.md filename

* lowcasing setup.md

* lowcasing configuration.md

* update reference to setup.md

* update reference to setup.md

* update reference to configuration.md

* lowcase bigquery ingestion guide filenames

* Update location of ingestion guides in sidebar

* renaming ingestion quickstart guide sidebar

* remove old files

* Update docs-website/sidebars.js

* tweak

Co-authored-by: Maggie Hays <maggiem.hays@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-12 18:43:43 -08:00

2.0 KiB
Raw Blame History

title
Overview

BigQuery Ingestion Guide: Overview

What You Will Get Out of This Guide

This guide will help you set up the BigQuery connector through the DataHub UI to begin ingesting metadata into DataHub.

Upon completing this guide, you will have a recurring ingestion pipeline that will extract metadata from BigQuery and load it into DataHub. This will include to following BigQuery asset types:

This recurring ingestion pipeline will also extract:

  • Usage statistics to help you understand recent query activity
  • Table-level lineage (where available) to automatically define interdependencies between datasets
  • Table- and column-level profile statistics to help you understand the shape of the data

:::caution You will NOT have extracted Routines, Search Indexes from BigQuery, as the connector does not support ingesting these assets :::

Next Steps

If that all sounds like what you're looking for, navigate to the next page, where we'll talk about prerequisites

Advanced Guides and Reference

If you're looking to do something more in-depth, want to use CLI instead of the DataHub UI, or just need to look at the reference documentation for this connector, use these links:

Need more help? Join the conversation in Slack!