yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-10-12 01:08:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

GitBook: [#93] New Changes

Browse Source
This commit is contained in:
Shilpa V 2022-03-14 16:13:06 +00:00 committed by Sriharsha Chintalapani
parent 1904263c3d
commit aa5dea4a86
6 changed files with 38 additions and 34 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

38
docs/README.md
View File

@ -10,25 +10,25 @@ OpenMetadata enables metadata management end-to-end, giving you the ability to u
OpenMetadata provides connectors that enable you to perform metadata ingestion from a number of common database, dashboard, messaging, and pipeline services. With each release, we add additional connectors and the ingestion framework provides a structured and straightforward method for creating your own connectors. See the table below for a list of supported connectors. OpenMetadata provides connectors that enable you to perform metadata ingestion from a number of common database, dashboard, messaging, and pipeline services. With each release, we add additional connectors and the ingestion framework provides a structured and straightforward method for creating your own connectors. See the table below for a list of supported connectors.
| A-H | I-M | N-R | S-Z | | A-H | I-M | N-R | S-Z |
| ---------------------------------------------------------------------------------------------- | ----------------------------------------------- | ----------------------------------------------------------- | ------------------------------------------------------------- | | ---------------------------------------------------------------------------------------------- | ----------------------------------------------- | -------------------------------------------------------------------- | ----------------------------------------------------------------------- |
| [Airflow](integrations/connectors/airflow/airflow.md) | [IBM Db2](integrations/connectors/ibm-db2.md) | [Oracle](integrations/connectors/oracle.md) | [Salesforce](integrations/connectors/salesforce.md) | | [Airflow](integrations/connectors/airflow/airflow.md) | [IBM Db2](integrations/connectors/ibm-db2.md) | [Oracle](integrations/connectors/oracle.md) | [Salesforce](integrations/connectors/salesforce.md) |
| Amundsen | [Kafka](integrations/connectors/kafka.md) | [Postgres](integrations/connectors/postgres/) | [SingleStore](integrations/connectors/singlestore.md) | | Amundsen | [Kafka](integrations/connectors/kafka.md) | [Postgres](integrations/connectors/postgres/) | [SingleStore](integrations/connectors/singlestore.md) |
| Apache Atlas | LDAP | Power BI | [Snowflake](integrations/connectors/snowflake/) | | Apache Atlas | LDAP | Power BI | [Snowflake](integrations/connectors/snowflake/) |
| Apache Druid | [Looker](integrations/connectors/looker.md) | Prefect | [Snowflake Usage](integrations/connectors/snowflake-usage.md) | | Apache Druid | [Looker](integrations/connectors/looker.md) | Prefect | [Snowflake Usage](integrations/connectors/snowflake/snowflake-usage.md) |
| [Athena](integrations/connectors/athena.md) | [MariaDB](integrations/connectors/mariadb.md) | [Presto](integrations/connectors/presto.md) | [Superset](integrations/connectors/superset.md) | | [Athena](integrations/connectors/athena.md) | [MariaDB](integrations/connectors/mariadb.md) | [Presto](integrations/connectors/presto.md) | [Superset](integrations/connectors/superset.md) |
| [Azure SQL](integrations/connectors/azure-sql.md) | [Metabase](integrations/connectors/metabase.md) | [Redash](integrations/connectors/redash.md) | [Tableau](integrations/connectors/tableau.md) | | [Azure SQL](integrations/connectors/azure-sql.md) | [Metabase](integrations/connectors/metabase.md) | [Redash](integrations/connectors/redash.md) | [Tableau](integrations/connectors/tableau.md) |
| [BigQuery](integrations/connectors/bigquery/) | [MLflow](integrations/connectors/mlflow/) | [Redshift](integrations/connectors/redshift/) | [Trino](integrations/connectors/trino.md) | | [BigQuery](integrations/connectors/bigquery/) | [MLflow](integrations/connectors/mlflow/) | [Redshift](integrations/connectors/redshift/) | [Trino](integrations/connectors/trino.md) |
| [BigQuery Usage](integrations/connectors/bigquery-usage.md) | [MSSQL](integrations/connectors/mssql/) | [Redshift Usage](integrations/connectors/redshift-usage.md) | [Vertica](integrations/connectors/vertica.md) | | [BigQuery Usage](integrations/connectors/bigquery/bigquery-usage.md) | [MSSQL](integrations/connectors/mssql/) | [Redshift Usage](integrations/connectors/redshift/redshift-usage.md) | [Vertica](integrations/connectors/vertica.md) |
| ClickHouse | MSSQL Usage | | | | ClickHouse | MSSQL Usage | | |
| ClickHouse Usage | [MySQL](integrations/connectors/mysql/mysql.md) | | | | ClickHouse Usage | [MySQL](integrations/connectors/mysql/mysql.md) | | |
| [Databricks](integrations/connectors/databricks.md) | | | | | [Databricks](integrations/connectors/databricks.md) | | | |
| [DBT](https://github.com/open-metadata/OpenMetadata/blob/main/docs/broken-reference/README.md) | | | | | [DBT](https://github.com/open-metadata/OpenMetadata/blob/main/docs/broken-reference/README.md) | | | |
| [Delta Lake](integrations/connectors/delta-lake.md) | | | | | [Delta Lake](integrations/connectors/delta-lake.md) | | | |
| [DynamoDB](integrations/connectors/dynamodb.md) | | | | | [DynamoDB](integrations/connectors/dynamodb.md) | | | |
| [Elasticsearch](integrations/connectors/elastic-search.md) | | | | | [Elasticsearch](integrations/connectors/elastic-search.md) | | | |
| [Glue Catalog](integrations/connectors/glue-catalog/) | | | | | [Glue Catalog](integrations/connectors/glue-catalog/) | | | |
| [Hive](integrations/connectors/hive.md) | | | | | [Hive](integrations/connectors/hive.md) | | | |
## OpenMetadata Components ## OpenMetadata Components

6
docs/SUMMARY.md
View File

@ -20,7 +20,7 @@
* [Azure SQL](integrations/connectors/azure-sql.md) * [Azure SQL](integrations/connectors/azure-sql.md)
* [BigQuery](integrations/connectors/bigquery/README.md) * [BigQuery](integrations/connectors/bigquery/README.md)
* [BigQuery Metadata Extraction](integrations/connectors/bigquery/bigquery-metadata-extraction.md) * [BigQuery Metadata Extraction](integrations/connectors/bigquery/bigquery-metadata-extraction.md)
* [BigQuery Usage](integrations/connectors/bigquery-usage.md) * [BigQuery Usage](integrations/connectors/bigquery/bigquery-usage.md)
* [Databricks](integrations/connectors/databricks.md) * [Databricks](integrations/connectors/databricks.md)
* [Delta Lake](integrations/connectors/delta-lake.md) * [Delta Lake](integrations/connectors/delta-lake.md)
* [DynamoDB](integrations/connectors/dynamodb.md) * [DynamoDB](integrations/connectors/dynamodb.md)
@ -46,12 +46,12 @@
* [Redash](integrations/connectors/redash.md) * [Redash](integrations/connectors/redash.md)
* [Redshift](integrations/connectors/redshift/README.md) * [Redshift](integrations/connectors/redshift/README.md)
* [Redshift Metadata Extraction](integrations/connectors/redshift/redshift-metadata-extraction.md) * [Redshift Metadata Extraction](integrations/connectors/redshift/redshift-metadata-extraction.md)
* [Redshift Usage](integrations/connectors/redshift-usage.md) * [Redshift Usage](integrations/connectors/redshift/redshift-usage.md)
* [Salesforce](integrations/connectors/salesforce.md) * [Salesforce](integrations/connectors/salesforce.md)
* [SingleStore](integrations/connectors/singlestore.md) * [SingleStore](integrations/connectors/singlestore.md)
* [Snowflake](integrations/connectors/snowflake/README.md) * [Snowflake](integrations/connectors/snowflake/README.md)
* [Snowflake Metadata Extraction](integrations/connectors/snowflake/snowflake-metadata-extraction.md) * [Snowflake Metadata Extraction](integrations/connectors/snowflake/snowflake-metadata-extraction.md)
* [Snowflake Usage](integrations/connectors/snowflake-usage.md) * [Snowflake Usage](integrations/connectors/snowflake/snowflake-usage.md)
* [Superset](integrations/connectors/superset.md) * [Superset](integrations/connectors/superset.md)
* [Tableau](integrations/connectors/tableau.md) * [Tableau](integrations/connectors/tableau.md)
* [Trino](integrations/connectors/trino.md) * [Trino](integrations/connectors/trino.md)

10
docs/integrations/connectors/README.md
View File

@ -4,10 +4,13 @@ OpenMetadata supports connectors to some popular services. We will continue as a
#### Database Services #### Database Services
* Apache Druid
* [Athena](athena.md) * [Athena](athena.md)
* [Azure SQL](azure-sql.md) * [Azure SQL](azure-sql.md)
* [BigQuery](bigquery/) * [BigQuery](bigquery/)
* [BigQuery Usage](bigquery-usage.md) * [BigQuery Usage](bigquery/bigquery-usage.md)
* ClickHouse
* ClickHouse Usage
* [Databricks](databricks.md) * [Databricks](databricks.md)
* [Delta Lake](delta-lake.md) * [Delta Lake](delta-lake.md)
* [DynamoDB](dynamodb.md) * [DynamoDB](dynamodb.md)
@ -22,11 +25,11 @@ OpenMetadata supports connectors to some popular services. We will continue as a
* [Postgres](postgres/) * [Postgres](postgres/)
* [Presto](presto.md) * [Presto](presto.md)
* [Redshift](redshift/) * [Redshift](redshift/)
* [Redshift Usage](redshift-usage.md) * [Redshift Usage](redshift/redshift-usage.md)
* [Salesforce](salesforce.md) * [Salesforce](salesforce.md)
* [SingleStore](singlestore.md) * [SingleStore](singlestore.md)
* [Snowflake](snowflake/) * [Snowflake](snowflake/)
* [Snowflake Usage](snowflake-usage.md) * [Snowflake Usage](snowflake/snowflake-usage.md)
* [Trino](trino.md) * [Trino](trino.md)
* [Vertica](vertica.md) * [Vertica](vertica.md)
@ -34,6 +37,7 @@ OpenMetadata supports connectors to some popular services. We will continue as a
* [Looker](looker.md) * [Looker](looker.md)
* [Metabase](metabase.md) * [Metabase](metabase.md)
* Power BI
* [Redash](redash.md) * [Redash](redash.md)
* [Superset](superset.md) * [Superset](superset.md)
* [Tableau](tableau.md) * [Tableau](tableau.md)

6
docs/integrations/connectors/bigquery-usage.md → docs/integrations/connectors/bigquery/bigquery-usage.md
View File

@ -356,7 +356,7 @@ The syntax and semantics for `schema_filter_pattern` are the same as for [`table
Use the `source.config.generate_sample_data` field to control whether or not to generate sample data to include in table views in the OpenMetadata user interface. The image below provides an example. Use the `source.config.generate_sample_data` field to control whether or not to generate sample data to include in table views in the OpenMetadata user interface. The image below provides an example.
![](../../.gitbook/assets/generate\_sample\_data.png) ![](../../../.gitbook/assets/generate\_sample\_data.png)
Explicitly include sample data by adding the following key-value pair in the `source.config` field of your configuration file. Explicitly include sample data by adding the following key-value pair in the `source.config` field of your configuration file.
@ -380,7 +380,7 @@ You can exclude the collection of sample data by adding the following key-value
DBT provides transformation logic that creates tables and views from raw data. OpenMetadata’s integration for DBT enables you to view the models used to generate a table from that table's details page in the OpenMetadata UI. The image below provides an example. DBT provides transformation logic that creates tables and views from raw data. OpenMetadata’s integration for DBT enables you to view the models used to generate a table from that table's details page in the OpenMetadata UI. The image below provides an example.
![](../../.gitbook/assets/configure\_dbt.png) ![](../../../.gitbook/assets/configure\_dbt.png)
To include DBT models and metadata in your ingestion workflows, specify the location of the DBT manifest and catalog files as fields in your configuration file. To include DBT models and metadata in your ingestion workflows, specify the location of the DBT manifest and catalog files as fields in your configuration file.
@ -441,7 +441,7 @@ metadata ingest -c ./bigquery_usage.json
As the ingestion workflow runs, you may observe progress both from the command line and from the OpenMetadata user interface. To view the metadata ingested from BigQuery Usage, visit [http://localhost:8585/explore/tables](http://localhost:8585/explore/tables). Select the BigQuery Usage service to filter for the data you’ve ingested using the workflow you configured and ran following this guide. The image below provides an example. As the ingestion workflow runs, you may observe progress both from the command line and from the OpenMetadata user interface. To view the metadata ingested from BigQuery Usage, visit [http://localhost:8585/explore/tables](http://localhost:8585/explore/tables). Select the BigQuery Usage service to filter for the data you’ve ingested using the workflow you configured and ran following this guide. The image below provides an example.
![](<../../.gitbook/assets/next\_steps (1).png>) ![](<../../../.gitbook/assets/next\_steps (1).png>)
## **Troubleshooting** ## **Troubleshooting**

6
docs/integrations/connectors/redshift-usage.md → docs/integrations/connectors/redshift/redshift-usage.md
View File

@ -324,7 +324,7 @@ The syntax and semantics for `schema_filter_pattern` are the same as for [`table
Use the `source.config.generate_sample_data` field to control whether or not to generate sample data to include in table views in the OpenMetadata user interface. The image below provides an example. Use the `source.config.generate_sample_data` field to control whether or not to generate sample data to include in table views in the OpenMetadata user interface. The image below provides an example.
![](../../.gitbook/assets/generate\_sample\_data.png) ![](../../../.gitbook/assets/generate\_sample\_data.png)
Explicitly include sample data by adding the following key-value pair in the `source.config` field of your configuration file. Explicitly include sample data by adding the following key-value pair in the `source.config` field of your configuration file.
@ -348,7 +348,7 @@ You can exclude the collection of sample data by adding the following key-value
DBT provides transformation logic that creates tables and views from raw data. OpenMetadata’s integration for DBT enables you to view the models used to generate a table from that table's details page in the OpenMetadata UI. The image below provides an example. DBT provides transformation logic that creates tables and views from raw data. OpenMetadata’s integration for DBT enables you to view the models used to generate a table from that table's details page in the OpenMetadata UI. The image below provides an example.
![](../../.gitbook/assets/configure\_dbt.png) ![](../../../.gitbook/assets/configure\_dbt.png)
To include DBT models and metadata in your ingestion workflows, specify the location of the DBT manifest and catalog files as fields in your configuration file. To include DBT models and metadata in your ingestion workflows, specify the location of the DBT manifest and catalog files as fields in your configuration file.
@ -409,7 +409,7 @@ metadata ingest -c ./redshift_usage.json
As the ingestion workflow runs, you may observe progress both from the command line and from the OpenMetadata user interface. To view the metadata ingested from Redshift Usage, visit [http://localhost:8585/explore/tables](http://localhost:8585/explore/tables). Select the Redshift Usage service to filter for the data you’ve ingested using the workflow you configured and ran following this guide. The image below provides an example. As the ingestion workflow runs, you may observe progress both from the command line and from the OpenMetadata user interface. To view the metadata ingested from Redshift Usage, visit [http://localhost:8585/explore/tables](http://localhost:8585/explore/tables). Select the Redshift Usage service to filter for the data you’ve ingested using the workflow you configured and ran following this guide. The image below provides an example.
![](<../../.gitbook/assets/next\_steps (1).png>) ![](<../../../.gitbook/assets/next\_steps (1).png>)
## **Troubleshooting** ## **Troubleshooting**

6
docs/integrations/connectors/snowflake-usage.md → docs/integrations/connectors/snowflake/snowflake-usage.md
View File

@ -324,7 +324,7 @@ The syntax and semantics for `schema_filter_pattern` are the same as for [`table
Use the `source.config.generate_sample_data` field to control whether or not to generate sample data to include in table views in the OpenMetadata user interface. The image below provides an example. Use the `source.config.generate_sample_data` field to control whether or not to generate sample data to include in table views in the OpenMetadata user interface. The image below provides an example.
![](../../.gitbook/assets/generate\_sample\_data.png) ![](../../../.gitbook/assets/generate\_sample\_data.png)
Explicitly include sample data by adding the following key-value pair in the `source.config` field of your configuration file. Explicitly include sample data by adding the following key-value pair in the `source.config` field of your configuration file.
@ -348,7 +348,7 @@ You can exclude the collection of sample data by adding the following key-value
DBT provides transformation logic that creates tables and views from raw data. OpenMetadata’s integration for DBT enables you to view the models used to generate a table from that table's details page in the OpenMetadata UI. The image below provides an example. DBT provides transformation logic that creates tables and views from raw data. OpenMetadata’s integration for DBT enables you to view the models used to generate a table from that table's details page in the OpenMetadata UI. The image below provides an example.
![](../../.gitbook/assets/configure\_dbt.png) ![](../../../.gitbook/assets/configure\_dbt.png)
To include DBT models and metadata in your ingestion workflows, specify the location of the DBT manifest and catalog files as fields in your configuration file. To include DBT models and metadata in your ingestion workflows, specify the location of the DBT manifest and catalog files as fields in your configuration file.
@ -409,7 +409,7 @@ metadata ingest -c ./snowflake_usage.json
As the ingestion workflow runs, you may observe progress both from the command line and from the OpenMetadata user interface. To view the metadata ingested from Snowflake Usage, visit [http://localhost:8585/explore/tables](http://localhost:8585/explore/tables). Select the Snowflake Usage service to filter for the data you’ve ingested using the workflow you configured and ran following this guide. The image below provides an example. As the ingestion workflow runs, you may observe progress both from the command line and from the OpenMetadata user interface. To view the metadata ingested from Snowflake Usage, visit [http://localhost:8585/explore/tables](http://localhost:8585/explore/tables). Select the Snowflake Usage service to filter for the data you’ve ingested using the workflow you configured and ran following this guide. The image below provides an example.
![](<../../.gitbook/assets/next\_steps (1).png>) ![](<../../../.gitbook/assets/next\_steps (1).png>)
## **Troubleshooting** ## **Troubleshooting**