docs: reformat use case guide toc & api comparison table (#7621)

Co-authored-by: Hyejin Yoon <yoonhyejin@ip-172-30-1-77.us-west-2.compute.internal>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: Hyejin Yoon <yoonhyejin@ip-192-168-0-10.us-west-2.compute.internal>
This commit is contained in:
Hyejin Yoon 2023-03-19 05:00:41 +09:00 committed by GitHub
parent 104c9811f5
commit b526dc1ab6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
25 changed files with 89 additions and 96 deletions

View File

@ -395,6 +395,33 @@ module.exports = {
},
],
},
{
"Getting Started: APIs & SDKs": [
{
"Creating Entities": [
"docs/api/tutorials/creating-datasets",
"docs/api/tutorials/creating-tags",
"docs/api/tutorials/creating-terms",
],
},
{
"Enriching Entities": [
"docs/api/tutorials/adding-tags",
"docs/api/tutorials/adding-terms",
"docs/api/tutorials/adding-ownerships",
"docs/api/tutorials/adding-dataset-description",
"docs/api/tutorials/adding-column-description",
"docs/api/tutorials/adding-lineage",
],
},
{
Reference: [
"docs/api/tutorials/references/generate-access-token",
"docs/api/tutorials/references/prepare-datahub",
],
},
],
},
],
},
@ -408,26 +435,6 @@ module.exports = {
],
},
"docs/datahub_lite",
{
"Getting Started: APIs & SDKs": [
"docs/tools/tutorials/advantages",
"docs/tools/tutorials/creating-datasets",
"docs/tools/tutorials/creating-tags",
"docs/tools/tutorials/creating-terms",
"docs/tools/tutorials/adding-tags",
"docs/tools/tutorials/adding-terms",
"docs/tools/tutorials/adding-ownerships",
"docs/tools/tutorials/adding-dataset-description",
"docs/tools/tutorials/adding-column-description",
"docs/tools/tutorials/adding-lineage",
{
Reference: [
"docs/tools/tutorials/references/generate-access-token",
"docs/tools/tutorials/references/prepare-datahub",
],
},
],
},
],
},
{

View File

@ -53,3 +53,28 @@ The Rest.li API represents the underlying persistence layer, and exposes the raw
href="/docs/api/restli/restli-overview">
Get started with our Rest.li API
</a>
## Datahub API Comparison
DataHub supports several APIs, each with its own unique usage and format.
Here's an overview of what each API can do.
> Last Updated : Mar 15 2023
| Feature | GraphQL | Python SDK | OpenAPI |
|---------------------------------------------------------|-----------------------------------------------------------------|----------------------------------------------------------------|---------|
| Create a dataset | 🚫 | ✅ [[Guide]](/docs/api/tutorials/creating-datasets.md) | ✅ |
| Create a tag | ✅ [[Guide]](/docs/api/tutorials/creating-tags.md) | ✅ [[Guide]](/docs/api/tutorials/creating-tags.md) | ✅ |
| Create a glossary term | ✅ [[Guide]](/docs/api/tutorials/creating-terms.md) | ✅ [[Guide]](/docs/api/tutorials/creating-terms.md) | ✅ |
| Add tags to a column of a dataset | ✅ [[Guide]](/docs/api/tutorials/adding-tags.md) | ✅ [[Guide]](/docs/api/tutorials/adding-tags.md) | ✅ |
| Add terms to a column of a dataset | ✅ [[Guide]](/docs/api/tutorials/adding-terms.md) | ✅ [[Guide]](/docs/api/tutorials/adding-terms.md) | ✅ |
| Add terms to a column of a dataset | ✅ [[Guide]](/docs/api/tutorials/adding-ownerships.md) | ✅ [[Guide]](/docs/api/tutorials/adding-ownerships.md) | ✅ |
| Add tags to a dataset | ✅ [[Guide]](/docs/api/tutorials/adding-tags.md) | ✅ | ✅ |
| Add terms to a dataset | ✅ [[Guide]](/docs/api/tutorials/adding-terms.md) | ✅ | ✅ |
| Add owner to a dataset | ✅ [[Guide]](/docs/api/tutorials/adding-ownerships.md) | ✅ | ✅ |
| Add lineage | ✅ [[Guide]](/docs/api/tutorials/adding-lineage.md) | ✅ [[Guide]](/docs/api/tutorials/adding-lineage.md) | ✅ |
| Add column level(Fine Grained) lineage | 🚫 | ✅ | ✅ |
| Add documentation(Description) to a column of a dataset | ✅ [[Guide]](/docs/api/tutorials/adding-column-description.md) | ✅ [[Guide]](/docs/api/tutorials/adding-column-description.md) | ✅ |
| Add documentation(Description) to a dataset | 🚫 | ✅ [[Guide]](/docs/api/tutorials/adding-dataset-description.md) | ✅ |
| Delete a dataset | 🚫 | ✅ | ✅ |
| Search a dataset | ✅ | ✅ | ✅ |

View File

@ -9,7 +9,7 @@ This guide will show you how to add a description to `user_name `column of a dat
## Prerequisites
For this tutorial, you need to deploy DataHub Quickstart and ingest sample data.
For detailed steps, please refer to [Prepare Local DataHub Environment](/docs/tools/tutorials/references/prepare-datahub.md).
For detailed steps, please refer to [Prepare Local DataHub Environment](/docs/api/tutorials/references/prepare-datahub.md).
:::note
Before adding a description, you need to ensure the targeted dataset is already present in your datahub.
@ -77,7 +77,7 @@ If you see the following response, the operation was successful:
### CURL
With CURL, you need to provide tokens. To generate a token, please refer to [Generate Access Token](/docs/tools/tutorials/references/generate-access-token.md).
With CURL, you need to provide tokens. To generate a token, please refer to [Generate Access Token](/docs/api/tutorials/references/generate-access-token.md).
With `accessToken`, you can run the following command.
```shell
@ -196,5 +196,5 @@ For more information about the `MetadataChangeProposal`, please refer to [Metada
## Expected Outcomes
You can now see column description is added to `user_name` column of `fct_users_deleted`.
![column-description-added](../../imgs/tutorials/column-description-added.png)
![column-description-added](../../imgs/apis/tutorials/column-description-added.png)

View File

@ -9,7 +9,7 @@ This guide will show you how to add a description and a link to dataset `fct_use
## Prerequisites
For this tutorial, you need to deploy DataHub Quickstart and ingest sample data.
For detailed steps, please refer to [Prepare Local DataHub Environment](/docs/tools/tutorials/references/prepare-datahub.md).
For detailed steps, please refer to [Prepare Local DataHub Environment](/docs/api/tutorials/references/prepare-datahub.md).
:::note
Before adding a description, you need to ensure the targeted dataset is already present in your datahub.
@ -21,7 +21,7 @@ In this example, we will add a description to `user_name `column of a dataset `f
## Add Description With GraphQL (Not Supported)
> 🚫 Adding Description on Dataset via GraphQL is currently not supported.
> Please check out [API feature comparison table](/docs/tools/tutorials/advantages.md#our-apis) for more information,
> Please check out [API feature comparison table](/docs/api/datahub-apis.md#datahub-api-comparison) for more information,
## Add Description With Python SDK
@ -134,5 +134,5 @@ For more information about the `MetadataChangeProposal`, please refer to [Metada
## Expected Outcomes
You can now see the description is added to `fct_users_deleted`.
![dataset-description-added](../../imgs/tutorials/dataset-description-added.png)
![dataset-description-added](../../imgs/apis/tutorials/dataset-description-added.png)

View File

@ -9,7 +9,7 @@ This guide will show you how to add lineage between two hive datasets named `fct
## Prerequisites
For this tutorial, you need to deploy DataHub Quickstart and ingest sample data.
For detailed steps, please refer to [Prepare Local DataHub Environment](/docs/tools/tutorials/references/prepare-datahub.md).
For detailed steps, please refer to [Prepare Local DataHub Environment](/docs/api/tutorials/references/prepare-datahub.md).
:::note
Before adding lineage, you need to ensure the targeted dataset is already present in your datahub.
@ -83,7 +83,7 @@ If you see the following response, the operation was successful:
### CURL
With CURL, you need to provide tokens. To generate a token, please refer to [Generate Access Token](/docs/tools/tutorials/references/generate-access-token.md).
With CURL, you need to provide tokens. To generate a token, please refer to [Generate Access Token](/docs/api/tutorials/references/generate-access-token.md).
With `accessToken`, you can run the following command.
```shell
@ -126,5 +126,5 @@ For more information about the `MetadataChangeEvent`, please refer to [Metadata
## Expected Outcomes
You can now see the lineage between `fct_users_deleted` and `logging_events`.
![lineage-added](../../imgs/tutorials/lineage-added.png)
![lineage-added](../../imgs/apis/tutorials/lineage-added.png)

View File

@ -10,7 +10,7 @@ This guide will show you how to add user group `bfoo` as an owner to the `fct_us
## Pre-requisites
For this tutorial, you need to deploy DataHub Quickstart and ingest sample data.
For detailed information, please refer to [Preparing Your Local DataHub Environment](/docs/tools/tutorials/references/prepare-datahub.md).
For detailed information, please refer to [Preparing Your Local DataHub Environment](/docs/api/tutorials/references/prepare-datahub.md).
:::note
Before adding owners, you need to ensure the targeted dataset and the owner are already present in your datahub.
@ -53,7 +53,7 @@ Expected Response:
### CURL
With CURL, you need to provide tokens. To generate a token, please refer to [Generate Access Token](/docs/tools/tutorials/references/generate-access-token.md).
With CURL, you need to provide tokens. To generate a token, please refer to [Generate Access Token](/docs/api/tutorials/references/generate-access-token.md).
With `accessToken`, you can run the following command.
```shell
@ -149,4 +149,4 @@ Expected Response:
## Expected Outcomes
You can now see `bfoo` has been added as an owner to the `fct_users_created` dataset.
![ownership-added](../../imgs/tutorials/owner-added.png)
![ownership-added](../../imgs/apis/tutorials/owner-added.png)

View File

@ -11,13 +11,13 @@ Additionally, we will cover how to add a tag to the dataset itself.
## Prerequisites
For this tutorial, you need to deploy DataHub Quickstart and ingest sample data.
For detailed steps, please refer to [Prepare Local DataHub Environment](/docs/tools/tutorials/references/prepare-datahub.md).
For detailed steps, please refer to [Prepare Local DataHub Environment](/docs/api/tutorials/references/prepare-datahub.md).
:::note
Before adding tags, you need to ensure the targeted dataset and the tag are already present in your datahub.
If you attempt to manipulate entities that do not exist, your operation will fail.
In this guide, we will be using data from a sample ingestion.
If you want to know how to create tags using APIs & SDKs, please refer to [Creating Tags](/docs/tools/tutorials/creating-tags.md) and [Creating Datasets](/docs/tools/tutorials/creating-datasets.md).
If you want to know how to create tags using APIs & SDKs, please refer to [Creating Tags](/docs/api/tutorials/creating-tags.md) and [Creating Datasets](/docs/api/tutorials/creating-datasets.md).
.
:::
@ -69,7 +69,7 @@ If you see the following response, the operation was successful:
### CURL
With CURL, you need to provide tokens. To generate a token, please refer to [Generate Access Token](/docs/tools/tutorials/references/generate-access-token.md).
With CURL, you need to provide tokens. To generate a token, please refer to [Generate Access Token](/docs/api/tutorials/references/generate-access-token.md).
With `accessToken`, you can run the following command.
```shell
@ -201,5 +201,5 @@ For more information about the `MetadataChangeProposal`, please refer to [Metada
## Expected Outcomes
You can now see `CustomerAccount` tag has been added to `user_name` column.
![tag-added](../../imgs/tutorials/tag-added.png)
![tag-added](../../imgs/apis/tutorials/tag-added.png)

View File

@ -12,13 +12,13 @@ Also, we will cover how to add a term to a dataset itself.
## Pre-requisites
For this tutorial, you need to deploy DataHub Quickstart and ingest sample data.
For detailed information, please refer to [Prepare Local DataHub Environment](/docs/tools/tutorials/references/prepare-datahub.md).
For detailed information, please refer to [Prepare Local DataHub Environment](/docs/api/tutorials/references/prepare-datahub.md).
:::note
Before adding terms, you need to ensure the targeted dataset and the term are already present in your datahub.
If you attempt to manipulate entities that do not exist, your operation will fail.
In this guide, we will be using data from a sample ingestion.
If you want to know how to create entities using APIs & SDKs, please refer to [Creating Terms](/docs/tools/tutorials/creating-terms.md) and [Creating Datasets](/docs/tools/tutorials/creating-datasets.md).
If you want to know how to create entities using APIs & SDKs, please refer to [Creating Terms](/docs/api/tutorials/creating-terms.md) and [Creating Datasets](/docs/api/tutorials/creating-datasets.md).
:::
@ -68,7 +68,7 @@ If you see the following response, the operation was successful:
### CURL
With CURL, you need to provide tokens. To generate a token, please refer to [Generate Access Token](/docs/tools/tutorials/references/generate-access-token.md).
With CURL, you need to provide tokens. To generate a token, please refer to [Generate Access Token](/docs/api/tutorials/references/generate-access-token.md).
With `accessToken`, you can run the following command.
```shell
@ -205,5 +205,5 @@ For more information about the `MetadataChangeProposal`, please refer to [Metada
## Expected Outcomes
You can now see the term `CustomerAccount` has been added to `user_name` column.
![term-added](../../imgs/tutorials/term-created.png)
![term-added](../../imgs/apis/tutorials/term-created.png)

View File

@ -9,12 +9,12 @@ This guide will show you how to create a dataset named `realestate_db.sales` wit
## Prerequisites
For this tutorial, you need to deploy DataHub Quickstart and ingest sample data.
For detailed steps, please refer to [Prepare Local DataHub Environment](/docs/tools/tutorials/references/prepare-datahub.md).
For detailed steps, please refer to [Prepare Local DataHub Environment](/docs/api/tutorials/references/prepare-datahub.md).
## Create Datasets With GraphQL (Not Supported)
> 🚫 Creating a dataset via GraphQL is currently not supported.
> Please check out [API feature comparison table](/docs/tools/tutorials/advantages.md#our-apis) for more information,
> Please check out [API feature comparison table](/docs/api/datahub-apis.md#datahub-api-comparison) for more information,
## Create Datasets With Python SDK
@ -98,16 +98,16 @@ For more information about the `MetadataChangeProposal`, please refer to [Metada
## Expected Outcomes
You can now see `realestate_db.sales` dataset has been created.
![dataset-created](../../imgs/tutorials/dataset-created.png)
![dataset-created](../../imgs/apis/tutorials/dataset-created.png)
## What's Next?
Now that you created a dataset, how about enriching it? Here are some guides that you can check out.
* [how to add a tag on a dataset](/docs/tools/tutorials/adding-tags.md).
* [how to add a term on a dataset](/docs/tools/tutorials/adding-terms.md).
* [how to add owner on a dataset](/docs/tools/tutorials/adding-ownerships.md).
* [how to add lineage on a dataset](/docs/tools/tutorials/adding-lineage.md).
* [how to add a tag on a dataset](/docs/api/tutorials/adding-tags.md).
* [how to add a term on a dataset](/docs/api/tutorials/adding-terms.md).
* [how to add owner on a dataset](/docs/api/tutorials/adding-ownerships.md).
* [how to add lineage on a dataset](/docs/api/tutorials/adding-lineage.md).

View File

@ -10,7 +10,7 @@ This guide will show you how to create a tag named `Deprecated`.
## Prerequisites
For this tutorial, you need to deploy DataHub Quickstart and ingest sample data.
For detailed steps, please refer to [Prepare Local DataHub Environment](/docs/tools/tutorials/references/prepare-datahub.md).
For detailed steps, please refer to [Prepare Local DataHub Environment](/docs/api/tutorials/references/prepare-datahub.md).
## Create Tags With GraphQL
@ -44,7 +44,7 @@ If you see the following response, the operation was successful:
### CURL
With CURL, you need to provide tokens. To generate a token, please refer to [Generate Access Token](/docs/tools/tutorials/references/generate-access-token.md).
With CURL, you need to provide tokens. To generate a token, please refer to [Generate Access Token](/docs/api/tutorials/references/generate-access-token.md).
With `accessToken`, you can run the following command.
```shell
@ -100,10 +100,10 @@ For more information about the `MetadataChangeProposal`, please refer to [Metada
## Expected Outcomes
You can now see `Deprecated` tag has been created.
![tag-created](../../imgs/tutorials/tag-created.png)
![tag-created](../../imgs/apis/tutorials/tag-created.png)
## What's Next?
Now that you created a tag, how about adding it to a dataset? Here's a guide on [how to add a tag on a dataset](/docs/tools/tutorials/adding-tags.md).
Now that you created a tag, how about adding it to a dataset? Here's a guide on [how to add a tag on a dataset](/docs/api/tutorials/adding-tags.md).

View File

@ -10,7 +10,7 @@ This guide will show you how to create a term named `Rate of Return`.
## Prerequisites
For this tutorial, you need to deploy DataHub Quickstart and ingest sample data.
For detailed steps, please refer to [Prepare Local DataHub Environment](/docs/tools/tutorials/references/prepare-datahub.md).
For detailed steps, please refer to [Prepare Local DataHub Environment](/docs/api/tutorials/references/prepare-datahub.md).
## Create Terms With GraphQL
@ -44,7 +44,7 @@ If you see the following response, the operation was successful:
### CURL
With CURL, you need to provide tokens. To generate a token, please refer to [Generate Access Token](/docs/tools/tutorials/references/generate-access-token.md).
With CURL, you need to provide tokens. To generate a token, please refer to [Generate Access Token](/docs/api/tutorials/references/generate-access-token.md).
With `accessToken`, you can run the following command.
```shell
@ -102,10 +102,10 @@ For more information about the `MetadataChangeProposal`, please refer to [Metada
You can now see `Rate of Return` term has been created.
To view the definition, you can either click on 'Govern > Glossary' at the top right of the page or simply search for the term by name.
![term-added](../../imgs/tutorials/term-added.png)
![term-added](../../imgs/apis/tutorials/term-added.png)
## What's Next?
Now that you created a term, how about adding it to a dataset? Here's a guide on [how to add a term on a dataset](/docs/tools/tutorials/adding-terms.md).
Now that you created a term, how about adding it to a dataset? Here's a guide on [how to add a term on a dataset](/docs/api/tutorials/adding-terms.md).

View File

@ -22,7 +22,7 @@ datahub docker quickstart
```
After logging in with the default credential(`username: datahub / password: datahub`), you can see DataHub ready for you.
![datahub-main-ui](../../../imgs/tutorials/datahub-main-ui.png)
![datahub-main-ui](../../../imgs/apis/tutorials/datahub-main-ui.png)
Please refer to [DataHub Quickstart Guide](/docs/quickstart.md) for more information.
@ -34,6 +34,6 @@ If you already have data on your datahub, you might skip this part.
datahub docker ingest-sample-data
```
This will ingest various entities like datasets, terms and tags to your local DataHub.
![datahub-main-ui](../../../imgs/tutorials/sample-ingestion.png)
![datahub-main-ui](../../../imgs/apis/tutorials/sample-ingestion.png)
Now you're ready to start!

View File

Before

Width:  |  Height:  |  Size: 47 KiB

After

Width:  |  Height:  |  Size: 47 KiB

View File

Before

Width:  |  Height:  |  Size: 137 KiB

After

Width:  |  Height:  |  Size: 137 KiB

View File

Before

Width:  |  Height:  |  Size: 68 KiB

After

Width:  |  Height:  |  Size: 68 KiB

View File

Before

Width:  |  Height:  |  Size: 101 KiB

After

Width:  |  Height:  |  Size: 101 KiB

View File

Before

Width:  |  Height:  |  Size: 154 KiB

After

Width:  |  Height:  |  Size: 154 KiB

View File

Before

Width:  |  Height:  |  Size: 179 KiB

After

Width:  |  Height:  |  Size: 179 KiB

View File

Before

Width:  |  Height:  |  Size: 135 KiB

After

Width:  |  Height:  |  Size: 135 KiB

View File

Before

Width:  |  Height:  |  Size: 143 KiB

After

Width:  |  Height:  |  Size: 143 KiB

View File

Before

Width:  |  Height:  |  Size: 23 KiB

After

Width:  |  Height:  |  Size: 23 KiB

View File

Before

Width:  |  Height:  |  Size: 144 KiB

After

Width:  |  Height:  |  Size: 144 KiB

View File

Before

Width:  |  Height:  |  Size: 64 KiB

After

Width:  |  Height:  |  Size: 64 KiB

View File

@ -1,39 +0,0 @@
# Advantages of using SDKs and APIs
## When to use an API over the DataHub UI
Using programmatic ways to emit metadata to DataHub can provide a number of benefits over using the UI. One key advantage is automation, which can save time and effort by streamlining the process of emitting metadata. Additionally, programmatic ways of emitting metadata can offer greater flexibility and control over the metadata being emitted, enabling you to customize the metadata to meet your specific needs.
## Simple use-cases to get started with DataHub APIs
APIs offer a wide range of use cases when it comes to emitting metadata.
Below are some simple use-cases to get you started:
### Basic Usage
* [Adding Tags](./adding-tags.md)
* [Adding Terms](./adding-terms.md)
* [Adding Ownership](./adding-ownerships.md)
### Advanced Usage
Here our some examples of slightly more complex usage:
* Adding Tags on Entities Based on Entity Type
* Ingesting Entities from CSV Files
* Adding Column-level Lineage
## Our APIs
DataHub supports three APIs : GraphQL, SDKs and OpenAPI. Each method has different usage and format.
Here's an overview of what each API can do.
> Last Updated : Mar 15 2023
| Feature | GraphQL | SDK | OpenAPI |
|--------------------------------------------------------|--------|-----|---------|
| Add Tags/Terms/Ownership to a column of a dataset | ✅ | ✅ | ✅ |
| Add Tags/Terms/Ownership to a dataset | ✅ | ✅ | ✅ |
| Create a dataset | | ✅ | ✅ |
| Delete a dataset | | ✅ | ✅ |
| Search a dataset | ✅ | ✅ | ✅ |
| Add lineage | ✅ | ✅ | ✅ |
| Add column level(Fine Grained) lineage | | ✅ | ✅ |
| Add documentation(Description) to a column of a dataset |✅ | ✅ | ✅ |
| Add documentation(Description) to a dataset | | ✅ | ✅ |
| Create a tag |✅ | ✅ | ✅ |
| Create a glossary term |✅ | ✅ | ✅ |