yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-12-14 03:26:47 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(cli): align default sink env variables across all cli commands (#4739)

Browse Source
This commit is contained in:
Shirshanka Das 2022-04-25 10:19:17 -07:00 committed by GitHub
parent a518e3d13e
commit a55073036d
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 5 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
metadata-ingestion/README.md
View File

@ -43,8 +43,8 @@ The default sink that most of the ingestion systems and guides assume is the `da
A recipe is the main configuration file that puts it all together. It tells our ingestion scripts where to pull data from (source) and where to put it (sink). A recipe is the main configuration file that puts it all together. It tells our ingestion scripts where to pull data from (source) and where to put it (sink).
Since `acryl-datahub` version `>=0.8.33.2`, the default sink is assumed to be a DataHub REST endpoint: Since `acryl-datahub` version `>=0.8.33.2`, the default sink is assumed to be a DataHub REST endpoint:
- Hosted at "http://localhost:8080" or the environment variable `${DATAHUB_HOST}` if present - Hosted at "http://localhost:8080" or the environment variable `${DATAHUB_GMS_HOST}` if present
- With an empty auth token or the environment variable `${DATAHUB_TOKEN}` if present. - With an empty auth token or the environment variable `${DATAHUB_GMS_TOKEN}` if present.
Here's a simple recipe that pulls metadata from MSSQL (source) and puts it into the default sink (datahub rest). Here's a simple recipe that pulls metadata from MSSQL (source) and puts it into the default sink (datahub rest).
@ -68,7 +68,7 @@ datahub ingest -c recipe.yaml
or if you want to override the default endpoints, you can provide the environment variables as part of the command like below: or if you want to override the default endpoints, you can provide the environment variables as part of the command like below:
```shell ```shell
DATAHUB_SERVER="https://my-datahub-server:8080" DATAHUB_TOKEN="my-datahub-token" datahub ingest -c recipe.yaml DATAHUB_GMS_HOST="https://my-datahub-server:8080" DATAHUB_GMS_TOKEN="my-datahub-token" datahub ingest -c recipe.yaml
``` ```
A number of recipes are included in the [examples/recipes](./examples/recipes) directory. For full info and context on each source and sink, see the pages described in the [table of plugins](../docs/cli.md#installing-plugins). A number of recipes are included in the [examples/recipes](./examples/recipes) directory. For full info and context on each source and sink, see the pages described in the [table of plugins](../docs/cli.md#installing-plugins).

4
metadata-ingestion/src/datahub/ingestion/run/pipeline.py
View File

@ -71,8 +71,8 @@ class PipelineConfig(ConfigModel):
default_sink_config = { default_sink_config = {
"type": "datahub-rest", "type": "datahub-rest",
"config": { "config": {
"server": "${DATAHUB_SERVER:-http://localhost:8080}", "server": "${DATAHUB_GMS_HOST:-http://localhost:8080}",
"token": "${DATAHUB_TOKEN:-}", "token": "${DATAHUB_GMS_TOKEN:-}",
}, },
} }
# resolve env variables if present # resolve env variables if present