yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-12-24 05:58:31 +00:00
Code Issues Packages Projects Releases Wiki Activity

updated metadata-ingestion and apis documents (#839)

Browse Source 
* updated metadata-ingestion and apis documents

* minor changes in

* addressing maven ci findings

Co-authored-by: Parth Panchal <parthpanchal@Parths-MacBook-Air.local>
This commit is contained in:
parthp2107 2021-10-18 18:02:22 +05:30 committed by GitHub
parent da6a57e2c1
commit aec586a5fe
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
37 changed files with 672 additions and 945 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
docs/SUMMARY.md
View File

@ -10,8 +10,8 @@
* [JSON Schema](openmetadata-apis/schemas/schema-language.md)
* [Schema Concepts](openmetadata-apis/schemas/overview.md)
* [OpenMetadata Types](openmetadata-apis/schemas/types/README.md)
* [Basic Types](openmetadata-apis/schemas/types/basic.md)
* [Audit Log](openmetadata-apis/schemas/types/auditlog.md)
* [Basic Types](openmetadata-apis/schemas/types/basic.md)
* [Daily Count](openmetadata-apis/schemas/types/dailycount.md)
* [Entity Lineage](openmetadata-apis/schemas/types/entitylineage.md)
* [Entity Reference](openmetadata-apis/schemas/types/entityreference.md)
@ -86,6 +86,7 @@
* [Oracle](install/metadata-ingestion/connectors/database-services/oracle.md)
* [Postgres](install/metadata-ingestion/connectors/database-services/postgres.md)
* [Presto](install/metadata-ingestion/connectors/database-services/presto.md)
* [Redshift](install/metadata-ingestion/connectors/database-services/redshift.md)
* [Redshift Usage](install/metadata-ingestion/connectors/database-services/redshift-usage.md)
* [Salesforce](install/metadata-ingestion/connectors/database-services/salesforce.md)
* [Snowflake](install/metadata-ingestion/connectors/database-services/snowflake.md)

8
docs/install/metadata-ingestion/connectors/database-services/README.md
View File

@ -34,6 +34,10 @@ OpenMetadata supports connectors to some popular data sources. We will continue
[postgres.md](postgres.md)
{% endcontent-ref %}
{% content-ref url="presto.md" %}
[presto.md](presto.md)
{% endcontent-ref %}
{% content-ref url="redshift.md" %}
[redshift.md](redshift.md)
{% endcontent-ref %}
@ -59,8 +63,12 @@ OpenMetadata supports connectors to some popular data sources. We will continue
{% endcontent-ref %}
{% page-ref page="trino.md" %}
[trino.md](trino.md)
{% endcontent-ref %}
{% page-ref page="vertica.md" %}
[vertica.md](vertica.md)
{% endcontent-ref %}

92
docs/install/metadata-ingestion/connectors/database-services/redshift.md Normal file
View File

@ -0,0 +1,92 @@
---
description: This guide will help install Redshift connector and run manually
---
# Redshift
{% hint style="info" %}
**Prerequisites**
OpenMetadata is built using Java, DropWizard, Jetty, and MySQL.
1. Python 3.7 or above
2. OpenMetadata Server up and running
{% endhint %}
## Install from PyPI <a href="install-from-pypi-or-source" id="install-from-pypi-or-source"></a>
```bash
pip install 'openmetadata-ingestion[redshift]'
```
## Run Manually <a href="run-manually" id="run-manually"></a>
```bash
metadata ingest -c ./examples/workflows/redshift.json
```
## Configuration
{% code title="redshift.json" %}
```javascript
{
"source": {
"type": "redshift",
"config": {
"host_port": "cluster.name.region.redshift.amazonaws.com:5439",
"username": "username",
"password": "strong_password",
"database": "warehouse",
"service_name": "aws_redshift",
"filter_pattern": {
"excludes": ["information_schema.*", "[\\w]*event_vw.*"]
}
}
},
...
```
{% endcode %}
1. **username** - pass the Redshift username.
2. **password** - the password for the Redshift username.
3. **service_name** - Service Name for this Redshift cluster. If you added the Redshift cluster through OpenMetadata UI, make sure the service name matches the same.
4. **filter_pattern** - It contains includes, excludes options to choose which pattern of datasets you want to ingest into OpenMetadata.
5. **database -** Database name from where data is to be fetched.‌
## Publish to OpenMetadata <a href="publish-to-openmetadata" id="publish-to-openmetadata"></a>
Below is the configuration to publish Redshift data into the OpenMetadata service.‌
Add Optionally `pii` processor and `metadata-rest` sink along with `metadata-server` config
{% code title="redshift.json" %}
```javascript
{
"source": {
"type": "redshift",
"config": {
"host_port": "cluster.name.region.redshift.amazonaws.com:5439",
"username": "username",
"password": "strong_password",
"database": "warehouse",
"service_name": "aws_redshift",
"filter_pattern": {
"excludes": ["information_schema.*", "[\\w]*event_vw.*"]
}
}
},
"sink": {
"type": "metadata-rest",
"config": {}
},
"metadata_server": {
"type": "metadata-server",
"config": {
"api_endpoint": "http://localhost:8585/api",
"auth_provider_type": "no-auth"
}
}
}
```
{% endcode %}

2
docs/open-source-community/developer/README.md
View File

@ -6,6 +6,8 @@ This document summarizes information relevant to OpenMetadata committers and con
{% page-ref page="coding-style.md" %}
{% page-ref page="how-to-contribute.md" %}
{% page-ref page="build-code-run-tests.md" %}
{% page-ref page="run-integration-tests.md" %}

28
docs/openmetadata-apis/schemas/entities/README.md
View File

@ -40,18 +40,22 @@ OpenMetadata has the following high-level core entities and relationships:
[messagingservice.md](messagingservice.md)
{% endcontent-ref %}
{% content-ref url="thread.md" %}
[thread.md](thread.md)
{% endcontent-ref %}
{% content-ref url="metrics.md" %}
[metrics.md](metrics.md)
{% endcontent-ref %}
{% content-ref url="model.md" %}
[model.md](model.md)
{% endcontent-ref %}
{% content-ref url="pipeline.md" %}
[pipeline.md](pipeline.md)
{% endcontent-ref %}
{% content-ref url="pipelineservice.md" %}
[pipelineservice.md](pipelineservice.md)
{% endcontent-ref %}
{% content-ref url="report.md" %}
[report.md](report.md)
{% endcontent-ref %}
@ -60,10 +64,22 @@ OpenMetadata has the following high-level core entities and relationships:
[table.md](table.md)
{% endcontent-ref %}
{% content-ref url="tagcategory.md" %}
[tagcategory.md](tagcategory.md)
{% endcontent-ref %}
{% content-ref url="task.md" %}
[task.md](task.md)
{% endcontent-ref %}
{% content-ref url="team.md" %}
[team.md](team.md)
{% endcontent-ref %}
{% content-ref url="thread.md" %}
[thread.md](thread.md)
{% endcontent-ref %}
{% content-ref url="topic.md" %}
[topic.md](topic.md)
{% endcontent-ref %}
@ -71,7 +87,3 @@ OpenMetadata has the following high-level core entities and relationships:
{% content-ref url="user.md" %}
[user.md](user.md)
{% endcontent-ref %}
{% content-ref url="tagcategory.md" %}
[tagcategory.md](tagcategory.md)
{% endcontent-ref %}

29
docs/openmetadata-apis/schemas/entities/bot.md
View File

@ -2,7 +2,7 @@
This schema defines Bot entity. A bot automates tasks, such as adding description, identifying the importance of data. It runs as a special user in the system.
**$id: **[https://open-metadata.org/schema/entity/bots.json](https://open-metadata.org/schema/entity/bots.json)
**$id:[https://open-metadata.org/schema/entity/bots.json](https://open-metadata.org/schema/entity/bots.json)**
Type: `object`
@ -20,25 +20,18 @@ Type: `object`
- **description**
- Description of the bot.
- Type: `string`
- **version**
- Metadata version of the entity.
- $ref: [../type/basic.json#/definitions/entityVersion](../types/basic.md#entityversion)
- **updatedAt**
- Last update time corresponding to the new version of the entity.
- $ref: [../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
- **updatedBy**
- User who made the update.
- Type: `string`
- **href**
- Link to the resource corresponding to this bot.
- $ref: [../type/basic.json#/definitions/href](../types/basic.md#href)
* **id**
* Unique identifier of a bot instance.
* $ref: [../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
* **name**
* Name of the bot.
* Type: `string`
* Length: between 1 and 64
* **displayName**
* Name used for display purposes. Example 'FirstName LastName'.
* Type: `string`
* **description**
* Description of the bot.
* Type: `string`
* **href**
* Link to the resource corresponding to this bot.
* $ref: [../type/basic.json#/definitions/href](../types/basic.md#href)
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

59
docs/openmetadata-apis/schemas/entities/chart.md
View File

@ -2,7 +2,7 @@
This schema defines the Chart entity. Charts are built using tables or sql queries by analyzing the data. Charts can be part of Dashboard.
**$id: **[https://open-metadata.org/schema/entity/data/chart.json](https://open-metadata.org/schema/entity/data/chart.json)
**$id: [https://open-metadata.org/schema/entity/data/chart.json](https://open-metadata.org/schema/entity/data/chart.json)**
Type: `object`
@ -24,6 +24,15 @@ Type: `object`
- **description**
- Description of the dashboard, what it is, and how to use it.
- Type: `string`
- **version**
- Metadata version of the entity.
- $ref: [../../type/basic.json#/definitions/entityVersion](../types/basic.md#entityversion)
- **updatedAt**
- Last update time corresponding to the new version of the entity.
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
- **updatedBy**
- User who made the update.
- Type: `string`
- **chartType**
- $ref: [#/definitions/chartType](#charttype)
- **chartUrl**
@ -54,52 +63,6 @@ Type: `object`
- Latest usage information for this database.
- $ref: [../../type/usageDetails.json](../types/usagedetails.md)
* **id** `required`
* Unique identifier that identifies a chart instance.
* $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
* **name** `required`
* Name that identifies this Chart.
* Type: `string`
* Length: between 1 and 64
* **displayName**
* Display Name that identifies this Chart. It could be a title or label from the source services.
* Type: `string`
* **fullyQualifiedName**
* A unique name that identifies a dashboard in the format 'ServiceName.ChartName'.
* Type: `string`
* Length: between 1 and 64
* **description**
* Description of the dashboard, what it is, and how to use it.
* Type: `string`
* **chartType**
* $ref: [#/definitions/chartType](chart.md#charttype)
* **chartUrl**
* Chart URL, pointing to its own Service URL.
* Type: `string`
* String format must be a "uri"
* **href**
* Link to the resource corresponding to this entity.
* $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
* **owner**
* Owner of this dashboard.
* $ref: [../../type/entityReference.json](../types/entityreference.md)
* **tables**
* Link to table used in this chart.
* $ref: [../../type/entityReference.json#/definitions/entityReferenceList](../types/entityreference.md#entityreferencelist)
* **followers**
* Followers of this chart.
* $ref: [../../type/entityReference.json#/definitions/entityReferenceList](../types/entityreference.md#entityreferencelist)
* **tags**
* Tags for this chart.
* Type: `array`
* **Items**
* $ref: [../../type/tagLabel.json](../types/taglabel.md)
* **service** `required`
* Link to service where this dashboard is hosted in.
* $ref: [../../type/entityReference.json](../types/entityreference.md)
* **usageSummary**
* Latest usage information for this database.
* $ref: [../../type/usageDetails.json](../types/usagedetails.md)
## Type definitions in this schema
### chartType
@ -119,4 +82,4 @@ Type: `object`
10. _"Other"_
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

59
docs/openmetadata-apis/schemas/entities/dashboard.md
View File

@ -2,7 +2,7 @@
This schema defines the Dashboard entity. Dashboards are computed from data and visually present data, metrics, and KPIs. They are updated in real-time and allow interactive data exploration.
**$id: **[https://open-metadata.org/schema/entity/data/dashboard.json](https://open-metadata.org/schema/entity/data/dashboard.json)
**$id: [https://open-metadata.org/schema/entity/data/dashboard.json](https://open-metadata.org/schema/entity/data/dashboard.json)**
Type: `object`
@ -24,6 +24,15 @@ Type: `object`
- **description**
- Description of the dashboard, what it is, and how to use it.
- Type: `string`
- **version**
- Metadata version of the entity.
- $ref: [../../type/basic.json#/definitions/entityVersion](../types/basic.md#entityversion)
- **updatedAt**
- Last update time corresponding to the new version of the entity.
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
- **updatedBy**
- User who made the update.
- Type: `string`
- **dashboardUrl**
- Dashboard URL.
- Type: `string`
@ -54,51 +63,5 @@ Type: `object`
- Latest usage information for this database.
- $ref: [../../type/usageDetails.json](../types/usagedetails.md)
* **id** `required`
* Unique identifier that identifies a dashboard instance.
* $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
* **name** `required`
* Name that identifies this dashboard.
* Type: `string`
* Length: between 1 and 64
* **displayName**
* Display Name that identifies this Dashboard. It could be a title or label from the source services.
* Type: `string`
* **fullyQualifiedName**
* A unique name that identifies a dashboard in the format 'ServiceName.DashboardName'.
* Type: `string`
* Length: between 1 and 64
* **description**
* Description of the dashboard, what it is, and how to use it.
* Type: `string`
* **dashboardUrl**
* Dashboard URL.
* Type: `string`
* String format must be a "uri"
* **charts**
* All the charts are included in this Dashboard.
* Type: `array`
* **Items**
* $ref: [../../type/entityReference.json](../types/entityreference.md)
* **href**
* Link to the resource corresponding to this entity.
* $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
* **owner**
* Owner of this dashboard.
* $ref: [../../type/entityReference.json](../types/entityreference.md)
* **followers**
* Followers of this dashboard.
* $ref: [../../type/entityReference.json#/definitions/entityReferenceList](../types/entityreference.md#entityreferencelist)
* **tags**
* Tags for this dashboard.
* Type: `array`
* **Items**
* $ref: [../../type/tagLabel.json](../types/taglabel.md)
* **service** `required`
* Link to service where this dashboard is hosted in.
* $ref: [../../type/entityReference.json](../types/entityreference.md)
* **usageSummary**
* Latest usage information for this database.
* $ref: [../../type/usageDetails.json](../types/usagedetails.md)
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

43
docs/openmetadata-apis/schemas/entities/dashboardservice.md
View File

@ -2,7 +2,7 @@
This schema defines the Dashboard Service entity, such as Looker and Superset.
**$id: **[https://open-metadata.org/schema/entity/services/dashboardService.json](https://open-metadata.org/schema/entity/services/dashboardService.json)
**$id: [https://open-metadata.org/schema/entity/services/dashboardService.json](https://open-metadata.org/schema/entity/services/dashboardService.json)**
Type: `object`
@ -23,6 +23,15 @@ Type: `object`
- **description**
- Description of a dashboard service instance.
- Type: `string`
- **version**
- Metadata version of the entity.
- $ref: [../../type/basic.json#/definitions/entityVersion](../types/basic.md#entityversion)
- **updatedAt**
- Last update time corresponding to the new version of the entity.
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
- **updatedBy**
- User who made the update.
- Type: `string`
- **dashboardUrl** `required`
- Dashboard Service URL. This will be used to make REST API calls to Dashboard Service.
- Type: `string`
@ -40,35 +49,6 @@ Type: `object`
- Link to the resource corresponding to this dashboard service.
- $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
* **id** `required`
* Unique identifier of this dashboard service instance.
* $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
* **name** `required`
* Name that identifies this dashboard service.
* Type: `string`
* Length: between 1 and 64
* **serviceType** `required`
* Type of dashboard services such as Looker or Superset...
* $ref: [#/definitions/dashboardServiceType](dashboardservice.md#dashboardservicetype)
* **description**
* Description of a dashboard service instance.
* Type: `string`
* **dashboardUrl** `required`
* Dashboard Service URL. This will be used to make REST API calls to Dashboard Service.
* Type: `string`
* String format must be a "uri"
* **username**
* Username to log in to Dashboard Service.
* Type: `string`
* **password**
* Password to log in to Dashboard Service.
* Type: `string`
* **ingestionSchedule**
* Schedule for running metadata ingestion jobs.
* $ref: [../../type/schedule.json](../types/schedule.md)
* **href**
* Link to the resource corresponding to this messaging service.
* $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
## Type definitions in this schema
### dashboardServiceType
@ -82,4 +62,5 @@ Type: `object`
4. _"Redash"_
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

41
docs/openmetadata-apis/schemas/entities/database.md
View File

@ -2,7 +2,7 @@
This schema defines the Database entity. A database also referred to as Database Catalog is a collection of tables.
**$id: **[https://open-metadata.org/schema/entity/data/database.json](https://open-metadata.org/schema/entity/data/database.json)
**$id: [https://open-metadata.org/schema/entity/data/database.json](https://open-metadata.org/schema/entity/data/database.json)**
Type: `object`
@ -22,6 +22,15 @@ Type: `object`
- **description**
- Description of the database instance.
- Type: `string`
- **version**
- Metadata version of the entity.
- $ref: [../../type/basic.json#/definitions/entityVersion](../types/basic.md#entityversion)
- **updatedAt**
- Last update time corresponding to the new version of the entity.
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
- **updatedBy**
- User who made the update.
- Type: `string`
- **href**
- Link to the resource corresponding to this entity.
- $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
@ -38,33 +47,6 @@ Type: `object`
- References to tables in the database.
- $ref: [../../type/entityReference.json#/definitions/entityReferenceList](../types/entityreference.md#entityreferencelist)
* **id**
* Unique identifier that identifies this database instance.
* $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
* **name** `required`
* Name that identifies the database.
* $ref: [#/definitions/databaseName](database.md#databasename)
* **fullyQualifiedName**
* Name that uniquely identifies a database in the format 'ServiceName.DatabaseName'.
* Type: `string`
* **description**
* Description of the database instance.
* Type: `string`
* **href**
* Link to the resource corresponding to this entity.
* $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
* **owner**
* Owner of this database.
* $ref: [../../type/entityReference.json](../types/entityreference.md)
* **service** `required`
* Link to the database cluster/service where this database is hosted in.
* $ref: [../../type/entityReference.json](../types/entityreference.md)
* **usageSummary**
* Latest usage information for this database.
* $ref: [../../type/usageDetails.json](../types/usagedetails.md)
* **tables**
* References to tables in the database.
* $ref: [../../type/entityReference.json#/definitions/entityReferenceList](../types/entityreference.md#entityreferencelist)
## Type definitions in this schema
### databaseName
@ -75,4 +57,5 @@ Type: `object`
- Length: between 1 and 64
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

44
docs/openmetadata-apis/schemas/entities/databaseservice.md
View File

@ -2,7 +2,7 @@
This schema defines the Database Service entity, such as MySQL, BigQuery, Redshift, Postgres, or Snowflake. Alternative terms such as Database Cluster, Database Server instance are also used for database service.
**$id: **[https://open-metadata.org/schema/entity/services/databaseService.json](https://open-metadata.org/schema/entity/services/databaseService.json)
**$id: [https://open-metadata.org/schema/entity/services/databaseService.json](https://open-metadata.org/schema/entity/services/databaseService.json)**
Type: `object`
@ -23,6 +23,15 @@ Type: `object`
- **description**
- Description of a database service instance.
- Type: `string`
- **version**
- Metadata version of the entity.
- $ref: [../../type/basic.json#/definitions/entityVersion](../types/basic.md#entityversion)
- **updatedAt**
- Last update time corresponding to the new version of the entity.
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
- **updatedBy**
- User who made the update.
- Type: `string`
- **href** `required`
- Link to the resource corresponding to this database service.
- $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
@ -33,35 +42,13 @@ Type: `object`
- Schedule for running metadata ingestion jobs.
- $ref: [../../type/schedule.json](../types/schedule.md)
* **id** `required`
* Unique identifier of this database service instance.
* $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
* **name** `required`
* Name that identifies this database service.
* Type: `string`
* Length: between 1 and 64
* **serviceType** `required`
* Type of database service such as MySQL, BigQuery, Snowflake, Redshift, Postgres...
* $ref: [#/definitions/databaseServiceType](databaseservice.md#databaseservicetype)
* **description**
* Description of a database service instance.
* Type: `string`
* **href** `required`
* Link to the resource corresponding to this database service.
* $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
* **jdbc** `required`
* JDBC connection information.
* $ref: [../../type/jdbcConnection.json#/definitions/jdbcInfo](../types/jdbcconnection.md#jdbcinfo)
* **ingestionSchedule**
* Schedule for running metadata ingestion jobs.
* $ref: [../../type/schedule.json](../types/schedule.md)
## Type definitions in this schema
### databaseServiceType
* Type of database service such as MySQL, BigQuery, Snowflake, Redshift, Postgres...
* Type: `string`
* The value is restricted to the following:
- Type of database service such as MySQL, BigQuery, Snowflake, Redshift, Postgres...
- Type: `string`
- The value is restricted to the following:
1. _"BigQuery"_
2. _"MySQL"_
3. _"Redshift"_
@ -72,6 +59,7 @@ Type: `object`
8. _"Oracle"_
9. _"Athena"_
10. _"Presto"_
11. _"Vertica"_
11. _"Trino"_
12. _"Vertica"_
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

48
docs/openmetadata-apis/schemas/entities/messagingservice.md
View File

@ -2,7 +2,7 @@
This schema defines the Messaging Service entity, such as Kafka and Pulsar.
**$id: **[https://open-metadata.org/schema/entity/services/messagingService.json](https://open-metadata.org/schema/entity/services/messagingService.json)
**$id: [https://open-metadata.org/schema/entity/services/messagingService.json](https://open-metadata.org/schema/entity/services/messagingService.json)**
Type: `object`
@ -21,8 +21,17 @@ Type: `object`
- Description of a messaging service instance.
- Type: `string`
- **displayName**
- Display Name that identifies this messaging service. It could be title or label from the source services.
- Type: `string`
- Display Name that identifies this messaging service. It could be title or label from the source services.
- Type: `string`
- **version**
- Metadata version of the entity.
- $ref: [../../type/basic.json#/definitions/entityVersion](../types/basic.md#entityversion)
- **updatedAt**
- Last update time corresponding to the new version of the entity.
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
- **updatedBy**
- User who made the update.
- Type: `string`
- **brokers** `required`
- Multiple bootstrap addresses for Kafka. Single proxy address for Pulsar.
- $ref: [#/definitions/brokers](#brokers)
@ -37,32 +46,6 @@ Type: `object`
- Link to the resource corresponding to this messaging service.
- $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
* **id** `required`
* Unique identifier of this messaging service instance.
* $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
* **name** `required`
* Name that identifies this messaging service.
* Type: `string`
* Length: between 1 and 64
* **serviceType** `required`
* Type of messaging services such as Kafka or Pulsar...
* $ref: [#/definitions/messagingServiceType](messagingservice.md#messagingservicetype)
* **description**
* Description of a messaging service instance.
* Type: `string`
* **brokers** `required`
* Multiple bootstrap addresses for Kafka. Single proxy address for Pulsar.
* $ref: [#/definitions/brokers](messagingservice.md#brokers)
* **schemaRegistry**
* Schema registry URL.
* Type: `string`
* String format must be a "uri"
* **ingestionSchedule**
* Schedule for running metadata ingestion jobs.
* $ref: [../../type/schedule.json](../types/schedule.md)
* **href**
* Link to the resource corresponding to this messaging service.
* $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
## Type definitions in this schema
### messagingServiceType
@ -76,4 +59,9 @@ Type: `object`
### brokers
_This document was updated on: Thursday, September 16, 2021_
- Multiple bootstrap addresses for Kafka. Single proxy address for Pulsar.
- Type: `array`
- **Items**
- Type: `string`
_This document was updated on: Monday, October 18, 2021_

39
docs/openmetadata-apis/schemas/entities/metrics.md
View File

@ -2,7 +2,7 @@
This schema defines the Metrics entity. Metrics are measurements computed from data such as `Monthly Active Users`. Some of the metrics that measures used to determine performance against an objective are called KPIs or Key Performance Indicators, such as `User Retention`.
**$id: **[https://open-metadata.org/schema/entity/data/metrics.json](https://open-metadata.org/schema/entity/data/metrics.json)
**$id: [https://open-metadata.org/schema/entity/data/metrics.json](https://open-metadata.org/schema/entity/data/metrics.json)**
Type: `object`
@ -24,6 +24,15 @@ Type: `object`
- **description**
- Description of metrics instance, what it is, and how to use it.
- Type: `string`
- **version**
- Metadata version of the entity.
- $ref: [../../type/basic.json#/definitions/entityVersion](../types/basic.md#entityversion)
- **updatedAt**
- Last update time corresponding to the new version of the entity.
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
- **updatedBy**
- User who made the update.
- Type: `string`
- **href**
- Link to the resource corresponding to this entity.
- $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
@ -37,31 +46,5 @@ Type: `object`
- Latest usage information for this database.
- $ref: [../../type/usageDetails.json](../types/usagedetails.md)
* **id** `required`
* Unique identifier that identifies this metrics instance.
* $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
* **name** `required`
* Name that identifies this metrics instance uniquely.
* Type: `string`
* Length: between 1 and 64
* **fullyQualifiedName**
* A unique name that identifies a metric in the format 'ServiceName.MetricName'.
* Type: `string`
* Length: between 1 and 64
* **description**
* Description of metrics instance, what it is, and how to use it.
* Type: `string`
* **href**
* Link to the resource corresponding to this entity.
* $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
* **owner**
* Owner of these metrics.
* $ref: [../../type/entityReference.json](../types/entityreference.md)
* **service** `required`
* Link to service where this metrics is hosted in.
* $ref: [../../type/entityReference.json](../types/entityreference.md)
* **usageSummary**
* Latest usage information for this database.
* $ref: [../../type/usageDetails.json](../types/usagedetails.md)
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

11
docs/openmetadata-apis/schemas/entities/model.md
View File

@ -21,6 +21,15 @@ Type: `object`
- **displayName**
- Display Name that identifies this model.
- Type: `string`
- **version**
- Metadata version of the entity.
- $ref: [../../type/basic.json#/definitions/entityVersion](../types/basic.md#entityversion)
- **updatedAt**
- Last update time corresponding to the new version of the entity.
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
- **updatedBy**
- User who made the update.
- Type: `string`
- **description**
- Description of the model, what it is, and how to use it.
- Type: `string`
@ -49,4 +58,4 @@ Type: `object`
- $ref: [../../type/usageDetails.json](../types/usagedetails.md)
_This document was updated on: Thursday, October 14, 2021_
_This document was updated on: Monday, October 18, 2021_

36
docs/openmetadata-apis/schemas/entities/pipeline.md
View File

@ -2,7 +2,7 @@
This schema defines the Pipeline entity. A pipeline enables the flow of data from source to destination through a series of processing steps. ETL is a type of pipeline where the series of steps Extract, Transform and Load the data.
**$id: **[https://open-metadata.org/schema/entity/data/pipeline.json](https://open-metadata.org/schema/entity/data/pipeline.json)
**$id: [https://open-metadata.org/schema/entity/data/pipeline.json](https://open-metadata.org/schema/entity/data/pipeline.json)**
Type: `object`
@ -24,6 +24,15 @@ Type: `object`
- **description**
- Description of this Pipeline.
- Type: `string`
- **version**
- Metadata version of the entity.
- $ref: [../../type/basic.json#/definitions/entityVersion](../types/basic.md#entityversion)
- **updatedAt**
- Last update time corresponding to the new version of the entity.
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
- **updatedBy**
- User who made the update.
- Type: `string`
- **pipelineUrl**
- Pipeline URL to visit/manage. This URL points to respective pipeline service UI.
- Type: `string`
@ -60,28 +69,5 @@ Type: `object`
- Link to service where this pipeline is hosted in.
- $ref: [../../type/entityReference.json](../types/entityreference.md)
* **id** `required`
* Unique identifier that identifies a pipeline instance.
* $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
* **name** `required`
* Name that identifies this pipeline instance uniquely.
* Type: `string`
* Length: between 1 and 64
* **fullyQualifiedName**
* A unique name that identifies a pipeline in the format 'ServiceName.PipelineName'.
* Type: `string`
* Length: between 1 and 64
* **description**
* Description of this pipeline.
* Type: `string`
* **href**
* Link to the resource corresponding to this entity.
* $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
* **owner**
* Owner of this pipeline.
* $ref: [../../type/entityReference.json](../types/entityreference.md)
* **service** `required`
* Link to service where this pipeline is hosted in.
* $ref: [../../type/entityReference.json](../types/entityreference.md)
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

16
docs/openmetadata-apis/schemas/entities/pipelineservice.md
View File

@ -21,8 +21,17 @@ Type: `object`
- Description of a pipeline service instance.
- Type: `string`
- **displayName**
- Display Name that identifies this pipeline service. It could be title or label from the source services.
- Type: `string`
- Display Name that identifies this pipeline service. It could be title or label from the source services.
- Type: `string`
- **version**
- Metadata version of the entity.
- $ref: [../../type/basic.json#/definitions/entityVersion](../types/basic.md#entityversion)
- **updatedAt**
- Last update time corresponding to the new version of the entity.
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
- **updatedBy**
- User who made the update.
- Type: `string`
- **pipelineUrl** `required`
- Pipeline Service Management/UI URL.
- Type: `string`
@ -44,5 +53,4 @@ Type: `object`
1. _"Airflow"_
2. _"Prefect"_
_This document was updated on: Thursday, October 14, 2021_
_This document was updated on: Monday, October 18, 2021_

39
docs/openmetadata-apis/schemas/entities/report.md
View File

@ -2,7 +2,7 @@
This schema defines the Report entity. Reports are static information computed from data periodically that includes data in text, table, and visual form.
**$id: **[https://open-metadata.org/schema/entity/data/report.json](https://open-metadata.org/schema/entity/data/report.json)
**$id: [https://open-metadata.org/schema/entity/data/report.json](https://open-metadata.org/schema/entity/data/report.json)**
Type: `object`
@ -24,6 +24,15 @@ Type: `object`
- **description**
- Description of this report instance.
- Type: `string`
- **version**
- Metadata version of the entity.
- $ref: [../../type/basic.json#/definitions/entityVersion](../types/basic.md#entityversion)
- **updatedAt**
- Last update time corresponding to the new version of the entity.
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
- **updatedBy**
- User who made the update.
- Type: `string`
- **href**
- Link to the resource corresponding to this report.
- $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
@ -37,31 +46,5 @@ Type: `object`
- Latest usage information for this database.
- $ref: [../../type/usageDetails.json](../types/usagedetails.md)
* **id** `required`
* The Unique identifier that identifies this report.
* $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
* **name** `required`
* Name that identifies this report instance uniquely.
* Type: `string`
* Length: between 1 and 64
* **fullyQualifiedName**
* A unique name that identifies a report in the format 'ServiceName.ReportName'.
* Type: `string`
* Length: between 1 and 64
* **description**
* Description of this report instance.
* Type: `string`
* **href**
* Link to the resource corresponding to this report.
* $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
* **owner**
* Owner of this pipeline.
* $ref: [../../type/entityReference.json](../types/entityreference.md)
* **service** `required`
* Link to service where this report is hosted in.
* $ref: [../../type/entityReference.json](../types/entityreference.md)
* **usageSummary**
* Latest usage information for this database.
* $ref: [../../type/usageDetails.json](../types/usagedetails.md)
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

447
docs/openmetadata-apis/schemas/entities/table.md
View File

@ -2,7 +2,7 @@
This schema defines the Table entity. A Table organizes data in rows and columns and is defined by a Schema. OpenMetadata does not have a separate abstraction for Schema. Both Table and Schema are captured in this entity.
**$id: **[https://open-metadata.org/schema/entity/data/table.json](https://open-metadata.org/schema/entity/data/table.json)
**$id: [https://open-metadata.org/schema/entity/data/table.json](https://open-metadata.org/schema/entity/data/table.json)**
Type: `object`
@ -14,19 +14,28 @@ Type: `object`
- Name of a table. Expected to be unique within a database.
- $ref: [#/definitions/tableName](#tablename)
- **displayName**
- Display Name that identifies this table. It could be title or label from the source services.
- Display Name that identifies this table. It could be title or label from the source services.
- Type: `string`
- **fullyQualifiedName**
- Fully qualified name of a table in the form `serviceName.databaseName.tableName`.
- Type: `string`
- **description**
- Description of a table.
- Type: `string`
- **version**
- Metadata version of the entity.
- $ref: [../../type/basic.json#/definitions/entityVersion](../types/basic.md#entityversion)
- **updatedAt**
- Last update time corresponding to the new version of the entity.
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
- **updatedBy**
- User who made the update.
- Type: `string`
- **href**
- Link to this table resource.
- $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
- **tableType**
- $ref: [#/definitions/tableType](#tabletype)
- **fullyQualifiedName**
- Fully qualified name of a table in the form `serviceName.databaseName.tableName`.
- Type: `string`
- **columns** `required`
- Columns in this table.
- Type: `array`
@ -37,15 +46,9 @@ Type: `object`
- Type: `array`
- **Items**
- $ref: [#/definitions/tableConstraint](#tableconstraint)
- **usageSummary**
- Latest usage information for this table.
- $ref: [../../type/usageDetails.json](../types/usagedetails.md)
- **owner**
- Owner of this table.
- $ref: [../../type/entityReference.json](../types/entityreference.md)
- **followers**
- Followers of this table.
- $ref: [../../type/entityReference.json#/definitions/entityReferenceList](../types/entityreference.md#entityreferencelist)
- **database**
- Reference to Database that contains this table.
- $ref: [../../type/entityReference.json](../types/entityreference.md)
@ -57,6 +60,12 @@ Type: `object`
- Type: `array`
- **Items**
- $ref: [../../type/tagLabel.json](../types/taglabel.md)
- **usageSummary**
- Latest usage information for this table.
- $ref: [../../type/usageDetails.json](../types/usagedetails.md)
- **followers**
- Followers of this table.
- $ref: [../../type/entityReference.json#/definitions/entityReferenceList](../types/entityreference.md#entityreferencelist)
- **joins**
- Details of other tables this table is frequently joined with.
- $ref: [#/definitions/tableJoins](#tablejoins)
@ -69,157 +78,105 @@ Type: `object`
- **Items**
- $ref: [#/definitions/tableProfile](#tableprofile)
* **id** `required`
* Unique identifier of this table instance.
* $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
* **name** `required`
* Name of a table. Expected to be unique within a database.
* $ref: [#/definitions/tableName](table.md#tablename)
* **description**
* Description of a table.
* Type: `string`
* **href**
* Link to this table resource.
* $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
* **tableType**
* $ref: [#/definitions/tableType](table.md#tabletype)
* **fullyQualifiedName**
* Fully qualified name of a table in the form `serviceName.databaseName.tableName`.
* Type: `string`
* **columns** `required`
* Columns in this table.
* Type: `array`
* **Items**
* $ref: [#/definitions/column](table.md#column)
* **tableConstraints**
* Table constraints.
* Type: `array`
* **Items**
* $ref: [#/definitions/tableConstraint](table.md#tableconstraint)
* **usageSummary**
* Latest usage information for this table.
* $ref: [../../type/usageDetails.json](../types/usagedetails.md)
* **owner**
* Owner of this table.
* $ref: [../../type/entityReference.json](../types/entityreference.md)
* **followers**
* Followers of this table.
* $ref: [../../type/entityReference.json#/definitions/entityReferenceList](../types/entityreference.md#entityreferencelist)
* **database**
* Reference to Database that contains this table.
* $ref: [../../type/entityReference.json](../types/entityreference.md)
* **viewDefinition**
* View Definition in SQL. Applies to TableType.View only.
* $ref: [../../type/basic.json#/definitions/sqlQuery](../types/basic.md#sqlquery)
* **tags**
* Tags for this table.
* Type: `array`
* **Items**
* $ref: [../../type/tagLabel.json](../types/taglabel.md)
* **joins**
* Details of other tables this table is frequently joined with.
* $ref: [#/definitions/tableJoins](table.md#tablejoins)
* **sampleData**
* Sample data for a table.
* $ref: [#/definitions/tableData](table.md#tabledata)
* **tableProfile**
* Data profile for a table.
* Type: `array`
* **Items**
* $ref: [#/definitions/tableProfile](table.md#tableprofile)
## Type definitions in this schema
### tableType
* This schema defines the type used for describing different types of tables.
* Type: `string`
* The value is restricted to the following:
1. _"Regular"_
2. _"External"_
3. _"View"_
4. _"SecureView"_
5. _"MaterializedView"_
- This schema defines the type used for describing different types of tables.
- Type: `string`
- The value is restricted to the following:
1. _"Regular"_
2. _"External"_
3. _"View"_
4. _"SecureView"_
5. _"MaterializedView"_
### columnDataType
* This enum defines the type of data stored in a column.
* Type: `string`
* The value is restricted to the following:
1. _"NUMBER"_
2. _"TINYINT"_
3. _"SMALLINT"_
4. _"INT"_
5. _"BIGINT"_
6. _"FLOAT"_
7. _"DOUBLE"_
8. _"DECIMAL"_
9. _"NUMERIC"_
10. _"TIMESTAMP"_
11. _"TIME"_
12. _"DATE"_
13. _"DATETIME"_
14. _"INTERVAL"_
15. _"STRING"_
16. _"MEDIUMTEXT"_
17. _"TEXT"_
18. _"CHAR"_
19. _"VARCHAR"_
20. _"BOOLEAN"_
21. _"BINARY"_
22. _"VARBINARY"_
23. _"ARRAY"_
24. _"BLOB"_
25. _"LONGBLOB"_
26. _"MEDIUMBLOB"_
27. _"MAP"_
28. _"STRUCT"_
29. _"UNION"_
30. _"SET"_
31. _"GEOGRAPHY"_
32. _"ENUM"_
33. _"JSON"_
### dataType
### columnConstraint
- This enum defines the type of data stored in a column.
- Type: `string`
- The value is restricted to the following:
1. _"NUMBER"_
2. _"TINYINT"_
3. _"SMALLINT"_
4. _"INT"_
5. _"BIGINT"_
6. _"BYTEINT"_
7. _"FLOAT"_
8. _"DOUBLE"_
9. _"DECIMAL"_
10. _"NUMERIC"_
11. _"TIMESTAMP"_
12. _"TIME"_
13. _"DATE"_
14. _"DATETIME"_
15. _"INTERVAL"_
16. _"STRING"_
17. _"MEDIUMTEXT"_
18. _"TEXT"_
19. _"CHAR"_
20. _"VARCHAR"_
21. _"BOOLEAN"_
22. _"BINARY"_
23. _"VARBINARY"_
24. _"ARRAY"_
25. _"BLOB"_
26. _"LONGBLOB"_
27. _"MEDIUMBLOB"_
28. _"MAP"_
29. _"STRUCT"_
30. _"UNION"_
31. _"SET"_
32. _"GEOGRAPHY"_
33. _"ENUM"_
34. _"JSON"_
### constraint
- This enum defines the type for column constraint.
- Type: `string`
- The value is restricted to the following:
1. _"NULL"_
2. _"NOT_NULL"_
3. _"UNIQUE"_
4. _"PRIMARY_KEY"_
* This enum defines the type for column constraint.
* Type: `string`
* The value is restricted to the following:
1. _"NULL"_
2. _"NOT_NULL"_
3. _"UNIQUE"_
4. _"PRIMARY_KEY"_
* Default: _"NULL"_
### tableConstraint
* This enum defines the type for table constraint.
* Type: `object`
* **Properties**
* **constraintType**
* Type: `string`
* The value is restricted to the following:
1. _"UNIQUE"_
2. _"PRIMARY_KEY"_
3. _"FOREIGN_KEY"_
* **columns**
* List of column names corresponding to the constraint.
* Type: `array`
* **Items**
* Type: `string`
- This enum defines the type for table constraint.
- Type: `object`
- **Properties**
- **constraintType**
- Type: `string`
- The value is restricted to the following:
1. _"UNIQUE"_
2. _"PRIMARY_KEY"_
3. _"FOREIGN_KEY"_
- **columns**
- List of column names corresponding to the constraint.
- Type: `array`
- **Items**
- Type: `string`
### columnName
* Local name (not fully qualified name) of the column.
* Type: `string`
* The value must match this pattern: `^[^.]*$`
* Length: between 1 and 64
- Local name (not fully qualified name) of the column. ColumnName is `-` when the column is not named in struct dataType. For example, BigQuery supports struct with unnamed fields.
- Type: `string`
- The value must match this pattern: `^[^.]*$`
- Length: between 1 and 64
### tableName
* Local name (not fully qualified name) of a table.
* Type: `string`
* The value must match this pattern: `^[^.]*$`
* Length: between 1 and 64
- Local name (not fully qualified name) of a table.
- Type: `string`
- The value must match this pattern: `^[^.]*$`
- Length: between 1 and 64
### fullyQualifiedColumnName
@ -230,84 +187,106 @@ Type: `object`
### column
* This schema defines the type for a column in a table.
* Type: `object`
* **Properties**
* **name** `required`
* $ref: [#/definitions/columnName](table.md#columnname)
* **columnDataType** `required`
* Data type of the column (int, date etc.).
* $ref: [#/definitions/columnDataType](table.md#columndatatype)
* **description**
* Description of the column.
* Type: `string`
* **fullyQualifiedName**
* $ref: [#/definitions/fullyQualifiedColumnName](table.md#fullyqualifiedcolumnname)
* **tags**
* Tags associated with the column.
* Type: `array`
* **Items**
* $ref: [../../type/tagLabel.json](../types/taglabel.md)
* **columnConstraint**
* Column level constraint.
* $ref: [#/definitions/columnConstraint](table.md#columnconstraint)
* **ordinalPosition**
* Ordinal position of the column.
* Type: `integer`
- This schema defines the type for a column in a table.
- Type: `object`
- This schema <u>does not</u> accept additional properties.
- **Properties**
- **name** `required`
- $ref: [#/definitions/columnName](#columnname)
- **dataType** `required`
- Data type of the column (int, date etc.).
- $ref: [#/definitions/dataType](#datatype)
- **arrayDataType**
- Data type used array in dataType. For example, `array<int>` has dataType as `array` and arrayDataType as `int`.
- $ref: [#/definitions/dataType](#datatype)
- **dataLength**
- Length of `char`, `varchar`, `binary`, `varbinary` `dataTypes`, else null. For example, `varchar(20)` has dataType as `varchar` and dataLength as `20`.
- Type: `integer`
- **dataTypeDisplay**
- Display name used for dataType. This is useful for complex types, such as `array<int>, map<int,string>, struct<>, and union types.
- Type: `string`
- **description**
- Description of the column.
- Type: `string`
- **fullyQualifiedName**
- $ref: [#/definitions/fullyQualifiedColumnName](#fullyqualifiedcolumnname)
- **tags**
- Tags associated with the column.
- Type: `array`
- **Items**
- $ref: [../../type/tagLabel.json](../types/taglabel.md)
- **constraint**
- Column level constraint.
- $ref: [#/definitions/constraint](#constraint)
- **ordinalPosition**
- Ordinal position of the column.
- Type: `integer`
- **jsonSchema**
- Json schema only if the dataType is JSON else null.
- Type: `string`
- **children**
- Child columns if dataType or arrayDataType is `map`, `struct`, or `union` else `null`.
- Type: `array`
- **Items**
- $ref: [#/definitions/column](#column)
### columnJoins
* This schema defines the type to capture how frequently a column is joined with columns in the other tables.
* Type: `object`
* This schema does not accept additional properties.
* **Properties**
* **columnName**
* $ref: [#/definitions/columnName](table.md#columnname)
* **joinedWith**
* Fully qualified names of the columns that this column is joined with.
* Type: `array`
* **Items**
* Type: `object`
* **Properties**
* **fullyQualifiedName**
* $ref: [#/definitions/fullyQualifiedColumnName](table.md#fullyqualifiedcolumnname)
* **joinCount**
* Type: `integer`
- This schema defines the type to capture how frequently a column is joined with columns in the other tables.
- Type: `object`
- This schema <u>does not</u> accept additional properties.
- **Properties**
- **columnName**
- $ref: [#/definitions/columnName](#columnname)
- **joinedWith**
- Fully qualified names of the columns that this column is joined with.
- Type: `array`
- **Items**
- Type: `object`
- **Properties**
- **fullyQualifiedName**
- $ref: [#/definitions/fullyQualifiedColumnName](#fullyqualifiedcolumnname)
- **joinCount**
- Type: `integer`
### tableJoins
* This schema defines the type to capture information about how columns in this table are joined with columns in the other tables.
* Type: `object`
* This schema does not accept additional properties.
* **Properties**
* **startDate**
* Date can be only from today going back to last 29 days.
* $ref: [../../type/basic.json#/definitions/date](../types/basic.md#date)
* **dayCount**
* Type: `integer`
* Default: `1`
* **columnJoins**
* Type: `array`
* **Items**
* $ref: [#/definitions/columnJoins](table.md#columnjoins)
- This schema defines the type to capture information about how columns in this table are joined with columns in the other tables.
- Type: `object`
- This schema <u>does not</u> accept additional properties.
- **Properties**
- **startDate**
- Date can be only from today going back to last 29 days.
- $ref: [../../type/basic.json#/definitions/date](../types/basic.md#date)
- **dayCount**
- Type: `integer`
- Default: `1`
- **columnJoins**
- Type: `array`
- **Items**
- $ref: [#/definitions/columnJoins](#columnjoins)
### tableData
* This schema defines the type to capture rows of sample data for a table.
* Type: `object`
* This schema does not accept additional properties.
* **Properties**
* **columns**
* List of local column names (not fully qualified column names) of the table.
* Type: `array`
* **Items**
* $ref: [#/definitions/columnName](table.md#columnname)
* **rows**
* Data for multiple rows of the table.
* Type: `array`
* **Items**
* Data for a single row of the table within the same order as columns fields.
* Type: `array`
- This schema defines the type to capture rows of sample data for a table.
- Type: `object`
- This schema <u>does not</u> accept additional properties.
- **Properties**
- **columns**
- List of local column names (not fully qualified column names) of the table.
- Type: `array`
- **Items**
- $ref: [#/definitions/columnName](#columnname)
- **rows**
- Data for multiple rows of the table.
- Type: `array`
- **Items**
- Data for a single row of the table within the same order as columns fields.
- Type: `array`
### columnProfile
@ -346,23 +325,25 @@ Type: `object`
- Type: `number`
* This schema defines the type to capture the table's data profile.
* Type: `object`
* This schema does not accept additional properties.
* **Properties**
* **profileDate**
* Data one which profile is taken.
* $ref: [../../type/basic.json#/definitions/date](../types/basic.md#date)
* **columnCount**
* No.of columns in the table.
* Type: `number`
* **rowCount**
* No.of rows in the table.
* Type: `number`
* **columnProfile**
* List of local column profiles of the table.
* Type: `array`
* **Items**
* $ref: [#/definitions/columnProfile](table.md#columnprofile)
### tableProfile
_This document was updated on: Thursday, September 16, 2021_
- This schema defines the type to capture the table's data profile.
- Type: `object`
- This schema <u>does not</u> accept additional properties.
- **Properties**
- **profileDate**
- Data one which profile is taken.
- $ref: [../../type/basic.json#/definitions/date](../types/basic.md#date)
- **columnCount**
- No.of columns in the table.
- Type: `number`
- **rowCount**
- No.of rows in the table.
- Type: `number`
- **columnProfile**
- List of local column profiles of the table.
- Type: `array`
- **Items**
- $ref: [#/definitions/columnProfile](#columnprofile)
_This document was updated on: Monday, October 18, 2021_

31
docs/openmetadata-apis/schemas/entities/tagcategory.md
View File

@ -2,7 +2,7 @@
This schema defines the Tag Category entity. A Tag Category contains tags called Primary Tags. Primary Tags can further have children Tags called Secondary Tags. Only two levels of tags are supported currently.
**$id: **[https://open-metadata.org/schema/entity/tags/tagCategory.json](https://open-metadata.org/schema/entity/tags/tagCategory.json)
**$id: [https://open-metadata.org/schema/entity/tags/tagCategory.json](https://open-metadata.org/schema/entity/tags/tagCategory.json)**
Type: `object`
@ -17,6 +17,15 @@ This schema <u>does not</u> accept additional properties.
- **description** `required`
- Description of the tag category.
- Type: `string`
- **version**
- Metadata version of the entity.
- $ref: [../../type/basic.json#/definitions/entityVersion](../types/basic.md#entityversion)
- **updatedAt**
- Last update time corresponding to the new version of the entity.
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
- **updatedBy**
- User who made the update.
- Type: `string`
- **categoryType** `required`
- $ref: [#/definitions/tagCategoryType](#tagcategorytype)
- **href**
@ -31,24 +40,6 @@ This schema <u>does not</u> accept additional properties.
- **Items**
- $ref: [#/definitions/tag](#tag)
* **name** `required`
* $ref: [#/definitions/tagName](tagcategory.md#tagname)
* **description** `required`
* Description of the tag category.
* Type: `string`
* **categoryType** `required`
* $ref: [#/definitions/tagCategoryType](tagcategory.md#tagcategorytype)
* **href**
* Link to the resource corresponding to the tag category.
* $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
* **usageCount**
* Count how many times the tags from this tag category are used.
* Type: `integer`
* **children**
* Tags under this category.
* Type: `array`
* **Items**
* $ref: [#/definitions/tag](tagcategory.md#tag)
## Type definitions in this schema
### tagName
@ -69,4 +60,4 @@ This schema <u>does not</u> accept additional properties.
### tag
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

11
docs/openmetadata-apis/schemas/entities/task.md
View File

@ -24,6 +24,15 @@ Type: `object`
- **description**
- Description of this Task.
- Type: `string`
- **version**
- Metadata version of the entity.
- $ref: [../../type/basic.json#/definitions/entityVersion](../types/basic.md#entityversion)
- **updatedAt**
- Last update time corresponding to the new version of the entity.
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
- **updatedBy**
- User who made the update.
- Type: `string`
- **taskUrl**
- Task URL to visit/manage. This URL points to respective pipeline service UI.
- Type: `string`
@ -62,4 +71,4 @@ Type: `object`
- $ref: [../../type/entityReference.json](../types/entityreference.md)
_This document was updated on: Thursday, October 14, 2021_
_This document was updated on: Monday, October 18, 2021_

38
docs/openmetadata-apis/schemas/entities/team.md
View File

@ -2,7 +2,7 @@
This schema defines the Team entity. A Team is a group of zero or more users. Teams can own zero or more data assets.
**$id: **[https://open-metadata.org/schema/entity/teams/team.json](https://open-metadata.org/schema/entity/teams/team.json)
**$id: [https://open-metadata.org/schema/entity/teams/team.json](https://open-metadata.org/schema/entity/teams/team.json)**
Type: `object`
@ -17,6 +17,15 @@ Type: `object`
- **description**
- Description of the team.
- Type: `string`
- **version**
- Metadata version of the entity.
- $ref: [../../type/basic.json#/definitions/entityVersion](../types/basic.md#entityversion)
- **updatedAt**
- Last update time corresponding to the new version of the entity.
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
- **updatedBy**
- User who made the update.
- Type: `string`
- **href** `required`
- Link to the resource corresponding to this entity.
- $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
@ -33,31 +42,6 @@ Type: `object`
- List of entities owned by the team.
- $ref: [../../type/entityReference.json#/definitions/entityReferenceList](../types/entityreference.md#entityreferencelist)
* **id** `required`
* $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
* **name** `required`
* $ref: [#/definitions/teamName](team.md#teamname)
* **displayName**
* Name used for display purposes. Example 'Data Science team'.
* Type: `string`
* **description**
* Description of the team.
* Type: `string`
* **href** `required`
* Link to the resource corresponding to this entity.
* $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
* **profile**
* Team profile information.
* $ref: [../../type/profile.json](../types/profile.md)
* **deleted**
* When true the team has been deleted.
* Type: `boolean`
* **users**
* Users that are part of the team.
* $ref: [../../type/entityReference.json#/definitions/entityReferenceList](../types/entityreference.md#entityreferencelist)
* **owns**
* List of entities owned by the team.
* $ref: [../../type/entityReference.json#/definitions/entityReferenceList](../types/entityreference.md#entityreferencelist)
## Type definitions in this schema
### teamName
@ -67,4 +51,4 @@ Type: `object`
- Length: between 1 and 64
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

50
docs/openmetadata-apis/schemas/entities/thread.md
View File

@ -2,7 +2,7 @@
This schema defines the Thread entity. A Thread is a collection of posts made by the users. The first post that starts a thread is **about** a data asset **from** a user. Other users can respond to this post by creating new posts in the thread. Note that bot users can also interact with a thread. A post can contains links that mention Users or other Data Assets.
**$id: **[https://open-metadata.org/schema/entity/feed/thread.json](https://open-metadata.org/schema/entity/feed/thread.json)
**$id: [https://open-metadata.org/schema/entity/feed/thread.json](https://open-metadata.org/schema/entity/feed/thread.json)**
Type: `object`
@ -27,41 +27,23 @@ Type: `object`
- **Items**
- $ref: [#/definitions/post](#post)
* **id** `required`
* Unique identifier that identifies an entity instance.
* $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
* **href**
* Link to the resource corresponding to this entity.
* $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
* **threadTs**
* Timestamp of the when the first post created the thread.
* $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
* **about** `required`
* Data asset about which this thread is created for with format <#E/{enties}/{entityName}/{field}/{fieldValue}.
* $ref: [../../type/basic.json#/definitions/entityLink](../types/basic.md#entitylink)
* **addressedTo**
* User or team this thread is addressed to in format <#E/{enties}/{entityName}/{field}/{fieldValue}.
* $ref: [../../type/basic.json#/definitions/entityLink](../types/basic.md#entitylink)
* **posts** `required`
* Type: `array`
* **Items**
* $ref: [#/definitions/post](thread.md#post)
## Type definitions in this schema
### post
* Post within a feed.
* Type: `object`
* **Properties**
* **message** `required`
* Message in markdown format. See markdown support for more details.
* Type: `string`
* **postTs**
* Timestamp of the post.
* Type: `string`
* String format must be a "date-time"
* **from** `required`
* The ID of the User (regular user or a bot) posting the message.
* $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
- Post within a feed.
- Type: `object`
- **Properties**
- **message** `required`
- Message in markdown format. See markdown support for more details.
- Type: `string`
- **postTs**
- Timestamp of the post.
- Type: `string`
- String format must be a "date-time"
- **from** `required`
- ID of User (regular user or a bot) posting the message.
- $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

77
docs/openmetadata-apis/schemas/entities/topic.md
View File

@ -2,7 +2,7 @@
This schema defines the Topic entity. A topic is a feed into which message are published to by publishers and read from by consumers in a messaging service.
**$id: **[https://open-metadata.org/schema/entity/data/topic.json](https://open-metadata.org/schema/entity/data/topic.json)
**$id: [https://open-metadata.org/schema/entity/data/topic.json](https://open-metadata.org/schema/entity/data/topic.json)**
Type: `object`
@ -17,11 +17,20 @@ Type: `object`
- Name that uniquely identifies a topic in the format 'messagingServiceName.topicName'.
- Type: `string`
- **displayName**
- Display Name that identifies this topic. It could be title or label from the source services.
- Type: `string`
- Display Name that identifies this topic. It could be title or label from the source services.
- Type: `string`
- **description**
- Description of the topic instance.
- Type: `string`
- **version**
- Metadata version of the entity.
- $ref: [../../type/basic.json#/definitions/entityVersion](../types/basic.md#entityversion)
- **updatedAt**
- Last update time corresponding to the new version of the entity.
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
- **updatedBy**
- User who made the update.
- Type: `string`
- **service** `required`
- Link to the messaging cluster/service where this topic is hosted in.
- $ref: [../../type/entityReference.json](../types/entityreference.md)
@ -71,66 +80,6 @@ Type: `object`
- Link to the resource corresponding to this entity.
- $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
* **id** `required`
* Unique identifier that identifies this topic instance.
* $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
* **name** `required`
* Name that identifies the topic.
* $ref: [#/definitions/topicName](topic.md#topicname)
* **fullyQualifiedName**
* Name that uniquely identifies a topic in the format 'messagingServiceName.topicName'.
* Type: `string`
* **description**
* Description of the topic instance.
* Type: `string`
* **service** `required`
* Link to the messaging cluster/service where this topic is hosted in.
* $ref: [../../type/entityReference.json](../types/entityreference.md)
* **partitions** `required`
* A number of partitions into which the topic is divided.
* Type: `integer`
* Range: ≥ 1
* **schemaText**
* The Schema is used for message serialization. Optional as some topics may not have associated schemas.
* Type: `string`
* **schemaType**
* Schema used for message serialization.
* $ref: [#/definitions/schemaType](topic.md#schematype)
* **cleanupPolicies**
* Topic clean up policies. For Kafka - `cleanup.policy` configuration.
* Type: `array`
* **Items**
* $ref: [#/definitions/cleanupPolicy](topic.md#cleanuppolicy)
* **retentionTime**
* Retention time in milliseconds. For Kafka - `retention.ms` configuration.
* Type: `number`
* **replicationFactor**
* Replication Factor in integer (more than 1).
* Type: `integer`
* **maximumMessageSize**
* Maximum message size in bytes. For Kafka - `max.message.bytes` configuration.
* Type: `integer`
* **minimumInSyncReplicas**
* The minimum number of replicas in sync to control durability. For Kafka - `min.insync.replicas` configuration.
* Type: `integer`
* **retentionSize**
* Maximum size of a partition in bytes before old data is discarded. For Kafka - `retention.bytes` configuration.
* Type: `number`
* Default: _"-1"_
* **owner**
* Owner of this topic.
* $ref: [../../type/entityReference.json](../types/entityreference.md)
* **followers**
* Followers of this table.
* $ref: [../../type/entityReference.json#/definitions/entityReferenceList](../types/entityreference.md#entityreferencelist)
* **tags**
* Tags for this table.
* Type: `array`
* **Items**
* $ref: [../../type/tagLabel.json](../types/taglabel.md)
* **href**
* Link to the resource corresponding to this entity.
* $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
## Type definitions in this schema
### topicName
@ -159,4 +108,4 @@ Type: `object`
2. _"compact"_
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

52
docs/openmetadata-apis/schemas/entities/user.md
View File

@ -2,7 +2,7 @@
This schema defines the User entity. A user can be part of 0 or more teams. A special type of user called Bot is used for automation. A user can be an owner of zero or more data assets. A user can also follow zero or more data assets.
**$id: **[https://open-metadata.org/schema/entity/teams/user.json](https://open-metadata.org/schema/entity/teams/user.json)
**$id: [https://open-metadata.org/schema/entity/teams/user.json](https://open-metadata.org/schema/entity/teams/user.json)**
Type: `object`
@ -17,6 +17,15 @@ This schema <u>does not</u> accept additional properties.
- **displayName**
- Name used for display purposes. Example 'FirstName LastName'.
- Type: `string`
- **version**
- Metadata version of the entity.
- $ref: [../../type/basic.json#/definitions/entityVersion](../types/basic.md#entityversion)
- **updatedAt**
- Last update time corresponding to the new version of the entity.
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
- **updatedBy**
- User who made the update.
- Type: `string`
- **email** `required`
- Email address of the user.
- $ref: [../../type/basic.json#/definitions/email](../types/basic.md#email)
@ -49,45 +58,6 @@ This schema <u>does not</u> accept additional properties.
- List of entities followed by the user.
- $ref: [../../type/entityReference.json#/definitions/entityReferenceList](../types/entityreference.md#entityreferencelist)
* **id** `required`
* Unique identifier that identifies a user entity instance.
* $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
* **name** `required`
* $ref: [#/definitions/userName](user.md#username)
* **displayName**
* Name used for display purposes. Example 'FirstName LastName'.
* Type: `string`
* **email** `required`
* Email address of the user.
* $ref: [../../type/basic.json#/definitions/email](../types/basic.md#email)
* **href** `required`
* Link to the resource corresponding to this entity.
* $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
* **timezone**
* The Timezone of the user.
* Type: `string`
* String format must be a "timezone"
* **deactivated**
* When true indicates the user has been deactivated. Users are deactivated instead of deleted.
* Type: `boolean`
* **isBot**
* When true indicates a special type of user called Bot.
* Type: `boolean`
* **isAdmin**
* When true indicates the user is an administrator for the system with superuser privileges.
* Type: `boolean`
* **profile**
* Profile of the user.
* $ref: [../../type/profile.json](../types/profile.md)
* **teams**
* Teams that the user belongs to.
* $ref: [../../type/entityReference.json#/definitions/entityReferenceList](../types/entityreference.md#entityreferencelist)
* **owns**
* List of entities owned by the user.
* $ref: [../../type/entityReference.json#/definitions/entityReferenceList](../types/entityreference.md#entityreferencelist)
* **follows**
* List of entities followed by the user.
* $ref: [../../type/entityReference.json#/definitions/entityReferenceList](../types/entityreference.md#entityreferencelist)
## Type definitions in this schema
### userName
@ -96,4 +66,4 @@ This schema <u>does not</u> accept additional properties.
- Type: `string`
- Length: between 1 and 64
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

6
docs/openmetadata-apis/schemas/types/README.md
View File

@ -2,10 +2,10 @@
OpenMetadata supports many common types reused across different schemas. Defining common types and reusing them is foundational to building a consistent schema vocabulary. The types are organized as follows:
{% page-ref page="basic.md" %}
{% page-ref page="auditlog.md" %}
{% page-ref page="basic.md" %}
{% page-ref page="dailycount.md" %}
{% page-ref page="entitylineage.md" %}
@ -22,5 +22,7 @@ OpenMetadata supports many common types reused across different schemas. Definin
{% page-ref page="schedule.md" %}
{% page-ref page="taglabel.md" %}
{% page-ref page="usagedetails.md" %}

31
docs/openmetadata-apis/schemas/types/auditlog.md
View File

@ -2,7 +2,7 @@
This schema defines the Audit Log type to capture the audit trail of POST, PUT, and PATCH API operations.
**$id: **[https://open-metadata.org/schema/type/auditLog.json](https://open-metadata.org/schema/type/auditLog.json)
**$id: [https://open-metadata.org/schema/type/auditLog.json](https://open-metadata.org/schema/type/auditLog.json)**
Type: `object`
@ -34,31 +34,4 @@ Type: `object`
- Type of Entity that is modified by the operation.
- Type: `string`
* **method** `required`
* HTTP Method used in a call.
* Type: `string`
* The value is restricted to the following:
1. _"POST"_
2. _"PUT"_
3. _"PATCH"_
4. _"DELETE"_
* **responseCode** `required`
* HTTP response code for the api request.
* Type: `integer`
* **path** `required`
* Requested API Path.
* Type: `string`
* **userName** `required`
* Name of the user who made the API request.
* Type: `string`
* **dateTime**
* Date when the API call is made.
* $ref: [basic.json#/definitions/dateTime](basic.md#datetime)
* **entityId** `required`
* Identifier of an entity that was modified by the operation.
* $ref: [basic.json#/definitions/uuid](basic.md#uuid)
* **entityType** `required`
* Type of Entity that is modified by the operation.
* Type: `string`
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

27
docs/openmetadata-apis/schemas/types/basic.md
View File

@ -2,7 +2,7 @@
This schema defines basic common types that are used by other schemas.
**$id: **[https://open-metadata.org/schema/type/basic.json](https://open-metadata.org/schema/type/basic.json)
**$id: [https://open-metadata.org/schema/type/basic.json](https://open-metadata.org/schema/type/basic.json)**
## Type definitions in this schema
@ -22,13 +22,6 @@ This schema defines basic common types that are used by other schemas.
- Length: between 6 and 127
### entityLink
- Link to an entity or field within an entity using this format `<#E/{enties}/{entityName}/{field}/{fieldValue}`.
- Type: `string`
- The value must match this pattern: `^<#E/\S+/\S+>$`
### timestamp
- Timestamp in unixTimeMillis.
@ -76,10 +69,26 @@ This schema defines basic common types that are used by other schemas.
- String format must be a "date-time"
### entityVersion
- Metadata version of the entity in the form `Major.Minor`. First version always starts from `0.1` when the entity is created. When the backward compatible changes are made to the entity, only the `Minor` version is incremented - example `1.0` is changed to `1.1`. When backward incompatible changes are made the `Major` version is incremented - example `1.1` to `2.0`.
- Type: `number`
- Default: `0.1`
- Range: &ge; 0.1
- The value must be a multiple of `0.1`
### entityLink
- Link to an entity or field within an entity using this format `<#E/{enties}/{entityName}/{field}/{fieldValue}`.
- Type: `string`
- The value must match this pattern: `^<#E/\S+/\S+>$`
### sqlQuery
- SQL query statement. Example - 'select * from orders'.
- Type: `string`
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

17
docs/openmetadata-apis/schemas/types/dailycount.md
View File

@ -2,19 +2,18 @@
This schema defines the type for reporting the daily count of some measurement. For example, you might use this schema for the number of times a table is queried each day.
**$id: **[https://open-metadata.org/schema/type/dailyCount.json](https://open-metadata.org/schema/type/dailyCount.json)
**$id: [https://open-metadata.org/schema/type/dailyCount.json](https://open-metadata.org/schema/type/dailyCount.json)**
Type: `object`
This schema <u>does not</u> accept additional properties.
## Properties
- **count** `required`
- Daily count of a measurement on the given date.
- Type: `integer`
- Range: &ge; 0
- **date** `required`
- $ref: [basic.json#/definitions/date](basic.md#date)
* **count** `required`
* Daily count of a measurement on the given date.
* Type: `integer`
* Range: ≥ 0
* **date** `required`
* $ref: [basic.json#/definitions/date](basic.md#date)
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

2
docs/openmetadata-apis/schemas/types/entitylineage.md
View File

@ -57,4 +57,4 @@ This schema <u>does not</u> accept additional properties.
- Type: `string`
_This document was updated on: Thursday, October 14, 2021_
_This document was updated on: Monday, October 18, 2021_

19
docs/openmetadata-apis/schemas/types/entityreference.md
View File

@ -2,7 +2,7 @@
This schema defines the EntityReference type used for referencing an entity. EntityReference is used for capturing relationships from one entity to another. For example, a table has an attribute called database of type EntityReference that captures the relationship of a table `belongs to a` database.
**$id: **[https://open-metadata.org/schema/type/entityReference.json](https://open-metadata.org/schema/type/entityReference.json)
**$id: [https://open-metadata.org/schema/type/entityReference.json](https://open-metadata.org/schema/type/entityReference.json)**
Type: `object`
@ -28,21 +28,6 @@ This schema <u>does not</u> accept additional properties.
- Link to the entity resource.
- $ref: [basic.json#/definitions/href](basic.md#href)
* **id** `required`
* Unique identifier that identifies an entity instance.
* $ref: [basic.json#/definitions/uuid](basic.md#uuid)
* **type** `required`
* Entity type/class name - Examples: `database`, `table`, `metrics`, `redshift`, `mysql`, `bigquery`, `snowflake`...
* Type: `string`
* **name**
* Name of the entity instance. For entities such as tables, databases where the name is not unique, fullyQualifiedName is returned in this field.
* Type: `string`
* **description**
* Optional description of the entity.
* Type: `string`
* **href**
* Link to the entity resource.
* $ref: [basic.json#/definitions/href](basic.md#href)
## Type definitions in this schema
### entityReferenceList
@ -52,4 +37,4 @@ This schema <u>does not</u> accept additional properties.
- $ref: [entityReference.json](entityreference.md)
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

13
docs/openmetadata-apis/schemas/types/entityusage.md
View File

@ -2,7 +2,7 @@
This schema defines the type used for capturing usage details of an entity.
**$id: **[https://open-metadata.org/schema/type/entityUsage.json](https://open-metadata.org/schema/type/entityUsage.json)
**$id: [https://open-metadata.org/schema/type/entityUsage.json](https://open-metadata.org/schema/type/entityUsage.json)**
Type: `object`
@ -16,13 +16,4 @@ Type: `object`
- **Items**
- $ref: [usageDetails.json](usagedetails.md)
* **entity** `required`
* Entity for which usage is returned.
* $ref: [entityReference.json](entityreference.md)
* **usage** `required`
* List usage details per day.
* Type: `array`
* **Items**
* $ref: [usageDetails.json](usagedetails.md)
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

30
docs/openmetadata-apis/schemas/types/jdbcconnection.md
View File

@ -2,7 +2,7 @@
This schema defines the type used for JDBC connection information.
**$id: **[https://open-metadata.org/schema/type/jdbcConnection.json](https://open-metadata.org/schema/type/jdbcConnection.json)
**$id: [https://open-metadata.org/schema/type/jdbcConnection.json](https://open-metadata.org/schema/type/jdbcConnection.json)**
Type: `object`
@ -20,18 +20,6 @@ Type: `object`
- Login password.
- Type: `string`
* **driverClass** `required`
* JDBC driver class.
* $ref: [#/definitions/driverClass](jdbcconnection.md#driverclass)
* **connectionUrl** `required`
* JDBC connection URL.
* $ref: [#/definitions/connectionUrl](jdbcconnection.md#connectionurl)
* **userName** `required`
* Login user name.
* Type: `string`
* **password** `required`
* Login password.
* Type: `string`
## Type definitions in this schema
### driverClass
@ -48,12 +36,12 @@ Type: `object`
### jdbcInfo
* Type for capturing JDBC connector information.
* Type: `object`
* **Properties**
* **driverClass** `required`
* $ref: [#/definitions/driverClass](jdbcconnection.md#driverclass)
* **connectionUrl** `required`
* $ref: [#/definitions/connectionUrl](jdbcconnection.md#connectionurl)
- Type for capturing JDBC connector information.
- Type: `object`
- **Properties**
- **driverClass** `required`
- $ref: [#/definitions/driverClass](#driverclass)
- **connectionUrl** `required`
- $ref: [#/definitions/connectionUrl](#connectionurl)
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

14
docs/openmetadata-apis/schemas/types/paging.md
View File

@ -2,7 +2,7 @@
Type used for cursor based pagination information in GET list responses.
**$id: **[https://open-metadata.org/schema/type/paging.json](https://open-metadata.org/schema/type/paging.json)
**$id: [https://open-metadata.org/schema/type/paging.json](https://open-metadata.org/schema/type/paging.json)**
Type: `object`
@ -17,14 +17,4 @@ Type: `object`
- Total number of entries available to page through.
- Type: `integer`
* **before**
* Before cursor is used for getting the previous page (see API pagination for details).
* Type: `string`
* **after**
* After the cursor is used for getting the next page (see API pagination for details).
* Type: `string`
* **total** `required`
* The Total number of entries available to page through.
* Type: `integer`
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

54
docs/openmetadata-apis/schemas/types/profile.md
View File

@ -2,7 +2,7 @@
This schema defines the type for a profile of a user, team, or organization.
**$id: **[https://open-metadata.org/schema/type/profile.json](https://open-metadata.org/schema/type/profile.json)
**$id: [https://open-metadata.org/schema/type/profile.json](https://open-metadata.org/schema/type/profile.json)**
Type: `object`
@ -10,35 +10,33 @@ Type: `object`
- **images**
- $ref: [#/definitions/imageList](#imagelist)
* **images**
* $ref: [#/definitions/imageList](profile.md#imagelist)
## Type definitions in this schema
### imageList
* Links to a list of images of varying resolutions/sizes.
* Type: `object`
* **Properties**
* **image**
* Type: `string`
* String format must be a "uri"
* **image24**
* Type: `string`
* String format must be a "uri"
* **image32**
* Type: `string`
* String format must be a "uri"
* **image48**
* Type: `string`
* String format must be a "uri"
* **image72**
* Type: `string`
* String format must be a "uri"
* **image192**
* Type: `string`
* String format must be a "uri"
* **image512**
* Type: `string`
* String format must be a "uri"
- Links to a list of images of varying resolutions/sizes.
- Type: `object`
- **Properties**
- **image**
- Type: `string`
- String format must be a "uri"
- **image24**
- Type: `string`
- String format must be a "uri"
- **image32**
- Type: `string`
- String format must be a "uri"
- **image48**
- Type: `string`
- String format must be a "uri"
- **image72**
- Type: `string`
- String format must be a "uri"
- **image192**
- Type: `string`
- String format must be a "uri"
- **image512**
- Type: `string`
- String format must be a "uri"
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

11
docs/openmetadata-apis/schemas/types/schedule.md
View File

@ -2,7 +2,7 @@
This schema defines the type used for the schedule. The schedule has a start time and repeat frequency.
**$id: **[https://open-metadata.org/schema/type/schedule.json](https://open-metadata.org/schema/type/schedule.json)
**$id: [https://open-metadata.org/schema/type/schedule.json](https://open-metadata.org/schema/type/schedule.json)**
Type: `object`
@ -14,11 +14,4 @@ Type: `object`
- Repeat frequency in ISO 8601 duration format. Example - 'P23DT23H'.
- $ref: [basic.json#/definitions/duration](basic.md#duration)
* **startDate**
* Start date and time of the schedule.
* $ref: [basic.json#/definitions/dateTime](basic.md#datetime)
* **repeatFrequency**
* Repeat frequency in ISO 8601 duration format. Example - 'P23DT23H'.
* $ref: [basic.json#/definitions/duration](basic.md#duration)
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

50
docs/openmetadata-apis/schemas/types/taglabel.md
View File

@ -2,33 +2,35 @@
This schema defines the type for labeling an entity with a Tag.
**$id: **[https://open-metadata.org/schema/type/tagLabel.json](https://open-metadata.org/schema/type/tagLabel.json)
**$id: [https://open-metadata.org/schema/type/tagLabel.json](https://open-metadata.org/schema/type/tagLabel.json)**
Type: `object`
This schema <u>does not</u> accept additional properties.
## Properties
- **tagFQN**
- Type: `string`
- Length: &le; 45
- **labelType**
- Label type describes how a tag label was applied. 'Manual' indicates the tag label was applied by a person. 'Derived' indicates a tag label was derived using the associated tag relationship (see TagCategory.json for more details). 'Propagated` indicates a tag label was propagated from upstream based on lineage. 'Automated' is used when a tool was used to determine the tag label.
- Type: `string`
- The value is restricted to the following:
1. _"Manual"_
2. _"Propagated"_
3. _"Automated"_
4. _"Derived"_
- Default: _"Manual"_
- **state**
- 'Suggested' state is used when a tag label is suggested by users or tools. Owner of the entity must confirm the suggested labels before it is marked as 'Confirmed'.
- Type: `string`
- The value is restricted to the following:
1. _"Suggested"_
2. _"Confirmed"_
- Default: _"Confirmed"_
- **href**
- Link to the tag resource.
- $ref: [basic.json#/definitions/href](basic.md#href)
* **tagFQN**
* Type: `string`
* Length: ≤ 45
* **labelType**
* Label type describes how a tag label was applied. 'Manual' indicates the tag label was applied by a person. 'Derived' indicates a tag label was derived using the associated tag relationship (see TagCategory.json for more details). 'Propagated\` indicates a tag label was propagated from upstream based on lineage. 'Automated' is used when a tool was used to determine the tag label.
* Type: `string`
* The value is restricted to the following:
1. _"Manual"_
2. _"Propagated"_
3. _"Automated"_
4. _"Derived"_
* Default: _"Manual"_
* **state**
* 'Suggested' state is used when a tag label is suggested by users or tools. The owner of the entity must confirm the suggested labels before it is marked as 'Confirmed'.
* Type: `string`
* The value is restricted to the following:
1. _"Suggested"_
2. _"Confirmed"_
* Default: _"Confirmed"_
* **href**
* Link to the tag resource.
* $ref: [basic.json#/definitions/href](basic.md#href)
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_

40
docs/openmetadata-apis/schemas/types/usagedetails.md
View File

@ -2,7 +2,7 @@
This schema defines the type for usage details. Daily, weekly, and monthly aggregation of usage is computed along with the percentile rank based on the usage for a given day.
**$id: **[https://open-metadata.org/schema/type/usageDetails.json](https://open-metadata.org/schema/type/usageDetails.json)
**$id: [https://open-metadata.org/schema/type/usageDetails.json](https://open-metadata.org/schema/type/usageDetails.json)**
Type: `object`
@ -20,33 +20,21 @@ Type: `object`
- Date in UTC.
- $ref: [basic.json#/definitions/date](basic.md#date)
* **dailyStats** `required`
* Daily usage stats of a data asset on the start date.
* $ref: [#/definitions/usageStats](usagedetails.md#usagestats)
* **weeklyStats**
* Weekly (last 7 days) rolling usage stats of a data asset on the start date.
* $ref: [#/definitions/usageStats](usagedetails.md#usagestats)
* **monthlyStats**
* Monthly (last 30 days) rolling usage stats of a data asset on the start date.
* $ref: [#/definitions/usageStats](usagedetails.md#usagestats)
* **date** `required`
* Date in UTC.
* $ref: [basic.json#/definitions/date](basic.md#date)
## Type definitions in this schema
### usageStats
* Type used to return usage statistics.
* Type: `object`
* This schema does not accept additional properties.
* **Properties**
* **count** `required`
* Usage count of a data asset on the start date.
* Type: `integer`
* Range: ≥ 0
* **percentileRank**
* Optional daily percentile rank data asset use when relevant.
* Type: `number`
* Range: between 0 and 100
- Type used to return usage statistics.
- Type: `object`
- This schema <u>does not</u> accept additional properties.
- **Properties**
- **count** `required`
- Usage count of a data asset on the start date.
- Type: `integer`
- Range: &ge; 0
- **percentileRank**
- Optional daily percentile rank data asset use when relevant.
- Type: `number`
- Range: between 0 and 100
_This document was updated on: Thursday, September 16, 2021_
_This document was updated on: Monday, October 18, 2021_