mirror of
https://github.com/open-metadata/OpenMetadata.git
synced 2025-10-14 10:18:23 +00:00
Updated doc with latest json schema (#743)
Co-authored-by: Parth Panchal <parthpanchal@Parths-MacBook-Air.local>
This commit is contained in:
parent
f0e59f2fea
commit
e4ac8499a7
@ -13,6 +13,7 @@
|
|||||||
* [Basic Types](openmetadata-apis/schemas/types/basic.md)
|
* [Basic Types](openmetadata-apis/schemas/types/basic.md)
|
||||||
* [Audit Log](openmetadata-apis/schemas/types/auditlog.md)
|
* [Audit Log](openmetadata-apis/schemas/types/auditlog.md)
|
||||||
* [Daily Count](openmetadata-apis/schemas/types/dailycount.md)
|
* [Daily Count](openmetadata-apis/schemas/types/dailycount.md)
|
||||||
|
* [Entity Lineage](openmetadata-apis/schemas/types/entitylineage.md)
|
||||||
* [Entity Reference](openmetadata-apis/schemas/types/entityreference.md)
|
* [Entity Reference](openmetadata-apis/schemas/types/entityreference.md)
|
||||||
* [Entity Usage](openmetadata-apis/schemas/types/entityusage.md)
|
* [Entity Usage](openmetadata-apis/schemas/types/entityusage.md)
|
||||||
* [Jdbc Connection](openmetadata-apis/schemas/types/jdbcconnection.md)
|
* [Jdbc Connection](openmetadata-apis/schemas/types/jdbcconnection.md)
|
||||||
@ -31,12 +32,15 @@
|
|||||||
* [Messaging Service](openmetadata-apis/schemas/entities/messagingservice.md)
|
* [Messaging Service](openmetadata-apis/schemas/entities/messagingservice.md)
|
||||||
* [Feed](openmetadata-apis/schemas/entities/thread.md)
|
* [Feed](openmetadata-apis/schemas/entities/thread.md)
|
||||||
* [Metrics](openmetadata-apis/schemas/entities/metrics.md)
|
* [Metrics](openmetadata-apis/schemas/entities/metrics.md)
|
||||||
|
* [Model](openmetadata-apis/schemas/entities/model.md)
|
||||||
* [Pipeline](openmetadata-apis/schemas/entities/pipeline.md)
|
* [Pipeline](openmetadata-apis/schemas/entities/pipeline.md)
|
||||||
|
* [Pipeline Service](openmetadata-apis/schemas/entities/pipelineservice.md)
|
||||||
* [Report](openmetadata-apis/schemas/entities/report.md)
|
* [Report](openmetadata-apis/schemas/entities/report.md)
|
||||||
* [Table](openmetadata-apis/schemas/entities/table.md)
|
* [Table](openmetadata-apis/schemas/entities/table.md)
|
||||||
|
* [Tag Category](openmetadata-apis/schemas/entities/tagcategory.md)
|
||||||
|
* [Task](openmetadata-apis/schemas/entities/task.md)
|
||||||
* [Team](openmetadata-apis/schemas/entities/team.md)
|
* [Team](openmetadata-apis/schemas/entities/team.md)
|
||||||
* [Topic](openmetadata-apis/schemas/entities/topic.md)
|
* [Topic](openmetadata-apis/schemas/entities/topic.md)
|
||||||
* [Tag Category](openmetadata-apis/schemas/entities/tagcategory.md)
|
|
||||||
* [User](openmetadata-apis/schemas/entities/user.md)
|
* [User](openmetadata-apis/schemas/entities/user.md)
|
||||||
* [APIs](openmetadata-apis/apis/README.md)
|
* [APIs](openmetadata-apis/apis/README.md)
|
||||||
* [Overview](openmetadata-apis/apis/overview.md)
|
* [Overview](openmetadata-apis/apis/overview.md)
|
||||||
|
@ -29,17 +29,22 @@ OpenMetadata has the following high-level core entities and relationships:
|
|||||||
|
|
||||||
{% page-ref page="metrics.md" %}
|
{% page-ref page="metrics.md" %}
|
||||||
|
|
||||||
|
{% page-ref page="model.md" %}
|
||||||
|
|
||||||
{% page-ref page="pipeline.md" %}
|
{% page-ref page="pipeline.md" %}
|
||||||
|
|
||||||
|
{% page-ref page="pipelineservice.md" %}
|
||||||
|
|
||||||
{% page-ref page="report.md" %}
|
{% page-ref page="report.md" %}
|
||||||
|
|
||||||
{% page-ref page="table.md" %}
|
{% page-ref page="table.md" %}
|
||||||
|
|
||||||
|
{% page-ref page="tagcategory.md" %}
|
||||||
|
|
||||||
|
{% page-ref page="task.md" %}
|
||||||
|
|
||||||
{% page-ref page="team.md" %}
|
{% page-ref page="team.md" %}
|
||||||
|
|
||||||
{% page-ref page="topic.md" %}
|
{% page-ref page="topic.md" %}
|
||||||
|
|
||||||
{% page-ref page="user.md" %}
|
{% page-ref page="user.md" %}
|
||||||
|
|
||||||
{% page-ref page="tagcategory.md" %}
|
|
||||||
|
|
||||||
|
@ -24,4 +24,4 @@ Type: `object`
|
|||||||
- Link to the resource corresponding to this bot.
|
- Link to the resource corresponding to this bot.
|
||||||
- $ref: [../type/basic.json#/definitions/href](../types/basic.md#href)
|
- $ref: [../type/basic.json#/definitions/href](../types/basic.md#href)
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -74,4 +74,4 @@ Type: `object`
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -54,4 +54,4 @@ Type: `object`
|
|||||||
- Latest usage information for this database.
|
- Latest usage information for this database.
|
||||||
- $ref: [../../type/usageDetails.json](../types/usagedetails.md)
|
- $ref: [../../type/usageDetails.json](../types/usagedetails.md)
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -51,4 +51,4 @@ Type: `object`
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -46,4 +46,4 @@ Type: `object`
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -47,7 +47,7 @@ Type: `object`
|
|||||||
8. _"Oracle"_
|
8. _"Oracle"_
|
||||||
9. _"Athena"_
|
9. _"Athena"_
|
||||||
10. _"Presto"_
|
10. _"Presto"_
|
||||||
11. _"Vertica"_
|
11. _"Trino"_
|
||||||
12. _"Trino"_
|
12. _"Vertica"_
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -54,4 +54,4 @@ Type: `object`
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -34,4 +34,4 @@ Type: `object`
|
|||||||
- Latest usage information for this database.
|
- Latest usage information for this database.
|
||||||
- $ref: [../../type/usageDetails.json](../types/usagedetails.md)
|
- $ref: [../../type/usageDetails.json](../types/usagedetails.md)
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
52
docs/openmetadata-apis/schemas/entities/model.md
Normal file
52
docs/openmetadata-apis/schemas/entities/model.md
Normal file
@ -0,0 +1,52 @@
|
|||||||
|
# Model
|
||||||
|
|
||||||
|
This schema defines the Model entity. Models are algorithms trained on data to find patterns or make predictions.
|
||||||
|
|
||||||
|
**$id: https://open-metadata.org/schema/entity/data/model.json**
|
||||||
|
|
||||||
|
Type: `object`
|
||||||
|
|
||||||
|
## Properties
|
||||||
|
- **id** `required`
|
||||||
|
- Unique identifier of a model instance.
|
||||||
|
- $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
|
||||||
|
- **name** `required`
|
||||||
|
- Name that identifies this model.
|
||||||
|
- Type: `string`
|
||||||
|
- Length: between 1 and 64
|
||||||
|
- **fullyQualifiedName**
|
||||||
|
- A unique name that identifies a model.
|
||||||
|
- Type: `string`
|
||||||
|
- Length: between 1 and 64
|
||||||
|
- **displayName**
|
||||||
|
- Display Name that identifies this model.
|
||||||
|
- Type: `string`
|
||||||
|
- **description**
|
||||||
|
- Description of the model, what it is, and how to use it.
|
||||||
|
- Type: `string`
|
||||||
|
- **algorithm** `required`
|
||||||
|
- Algorithm used to train the model.
|
||||||
|
- Type: `string`
|
||||||
|
- **dashboard**
|
||||||
|
- Performance Dashboard URL to track metric evolution.
|
||||||
|
- $ref: [../../type/entityReference.json](../types/entityreference.md)
|
||||||
|
- **href**
|
||||||
|
- Link to the resource corresponding to this entity.
|
||||||
|
- $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
|
||||||
|
- **owner**
|
||||||
|
- Owner of this model.
|
||||||
|
- $ref: [../../type/entityReference.json](../types/entityreference.md)
|
||||||
|
- **followers**
|
||||||
|
- Followers of this model.
|
||||||
|
- $ref: [../../type/entityReference.json#/definitions/entityReferenceList](../types/entityreference.md#entityreferencelist)
|
||||||
|
- **tags**
|
||||||
|
- Tags for this model.
|
||||||
|
- Type: `array`
|
||||||
|
- **Items**
|
||||||
|
- $ref: [../../type/tagLabel.json](../types/taglabel.md)
|
||||||
|
- **usageSummary**
|
||||||
|
- Latest usage information for this model.
|
||||||
|
- $ref: [../../type/usageDetails.json](../types/usagedetails.md)
|
||||||
|
|
||||||
|
|
||||||
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -14,13 +14,42 @@ Type: `object`
|
|||||||
- Name that identifies this pipeline instance uniquely.
|
- Name that identifies this pipeline instance uniquely.
|
||||||
- Type: `string`
|
- Type: `string`
|
||||||
- Length: between 1 and 64
|
- Length: between 1 and 64
|
||||||
|
- **displayName**
|
||||||
|
- Display Name that identifies this Pipeline. It could be title or label from the source services.
|
||||||
|
- Type: `string`
|
||||||
- **fullyQualifiedName**
|
- **fullyQualifiedName**
|
||||||
- A unique name that identifies a pipeline in the format 'ServiceName.PipelineName'.
|
- A unique name that identifies a pipeline in the format 'ServiceName.PipelineName'.
|
||||||
- Type: `string`
|
- Type: `string`
|
||||||
- Length: between 1 and 64
|
- Length: between 1 and 64
|
||||||
- **description**
|
- **description**
|
||||||
- Description of this pipeline.
|
- Description of this Pipeline.
|
||||||
- Type: `string`
|
- Type: `string`
|
||||||
|
- **pipelineUrl**
|
||||||
|
- Pipeline URL to visit/manage. This URL points to respective pipeline service UI.
|
||||||
|
- Type: `string`
|
||||||
|
- String format must be a "uri"
|
||||||
|
- **concurrency**
|
||||||
|
- Concurrency of the Pipeline.
|
||||||
|
- Type: `integer`
|
||||||
|
- **pipelineLocation**
|
||||||
|
- Pipeline Code Location.
|
||||||
|
- Type: `string`
|
||||||
|
- **startDate**
|
||||||
|
- Start date of the workflow.
|
||||||
|
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
|
||||||
|
- **tasks**
|
||||||
|
- All the tasks that are part of pipeline.
|
||||||
|
- Type: `array`
|
||||||
|
- **Items**
|
||||||
|
- $ref: [../../type/entityReference.json](../types/entityreference.md)
|
||||||
|
- **followers**
|
||||||
|
- Followers of this Pipeline.
|
||||||
|
- $ref: [../../type/entityReference.json#/definitions/entityReferenceList](../types/entityreference.md#entityreferencelist)
|
||||||
|
- **tags**
|
||||||
|
- Tags for this Pipeline.
|
||||||
|
- Type: `array`
|
||||||
|
- **Items**
|
||||||
|
- $ref: [../../type/tagLabel.json](../types/taglabel.md)
|
||||||
- **href**
|
- **href**
|
||||||
- Link to the resource corresponding to this entity.
|
- Link to the resource corresponding to this entity.
|
||||||
- $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
|
- $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
|
||||||
@ -31,4 +60,4 @@ Type: `object`
|
|||||||
- Link to service where this pipeline is hosted in.
|
- Link to service where this pipeline is hosted in.
|
||||||
- $ref: [../../type/entityReference.json](../types/entityreference.md)
|
- $ref: [../../type/entityReference.json](../types/entityreference.md)
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
45
docs/openmetadata-apis/schemas/entities/pipelineservice.md
Normal file
45
docs/openmetadata-apis/schemas/entities/pipelineservice.md
Normal file
@ -0,0 +1,45 @@
|
|||||||
|
# Pipeline Service
|
||||||
|
|
||||||
|
This schema defines the Pipeline Service entity, such as Airflow and Prefect.
|
||||||
|
|
||||||
|
**$id: https://open-metadata.org/schema/entity/services/messagingservice.json**
|
||||||
|
|
||||||
|
Type: `object`
|
||||||
|
|
||||||
|
## Properties
|
||||||
|
- **id** `required`
|
||||||
|
- Unique identifier of this pipeline service instance.
|
||||||
|
- $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
|
||||||
|
- **name** `required`
|
||||||
|
- Name that identifies this pipeline service.
|
||||||
|
- Type: `string`
|
||||||
|
- Length: between 1 and 64
|
||||||
|
- **serviceType**
|
||||||
|
- Type of pipeline service such as Airflow or Prefect...
|
||||||
|
- $ref: [#/definitions/pipelineServiceType](#pipelineservicetype)
|
||||||
|
- **description**
|
||||||
|
- Description of a pipeline service instance.
|
||||||
|
- Type: `string`
|
||||||
|
- **pipelineUrl** `required`
|
||||||
|
- Pipeline Service Management/UI URL.
|
||||||
|
- Type: `string`
|
||||||
|
- String format must be a "uri"
|
||||||
|
- **ingestionSchedule**
|
||||||
|
- Schedule for running metadata ingestion jobs.
|
||||||
|
- $ref: [../../type/schedule.json](../types/schedule.md)
|
||||||
|
- **href**
|
||||||
|
- Link to the resource corresponding to this pipeline service.
|
||||||
|
- $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
|
||||||
|
|
||||||
|
|
||||||
|
## Type definitions in this schema
|
||||||
|
### pipelineServiceType
|
||||||
|
|
||||||
|
- Type of pipeline service - Airflow or Prefect.
|
||||||
|
- Type: `string`
|
||||||
|
- The value is restricted to the following:
|
||||||
|
1. _"Airflow"_
|
||||||
|
2. _"Prefect"_
|
||||||
|
|
||||||
|
|
||||||
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -34,4 +34,4 @@ Type: `object`
|
|||||||
- Latest usage information for this database.
|
- Latest usage information for this database.
|
||||||
- $ref: [../../type/usageDetails.json](../types/usagedetails.md)
|
- $ref: [../../type/usageDetails.json](../types/usagedetails.md)
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -80,7 +80,7 @@ Type: `object`
|
|||||||
5. _"MaterializedView"_
|
5. _"MaterializedView"_
|
||||||
|
|
||||||
|
|
||||||
### columnDataType
|
### dataType
|
||||||
|
|
||||||
- This enum defines the type of data stored in a column.
|
- This enum defines the type of data stored in a column.
|
||||||
- Type: `string`
|
- Type: `string`
|
||||||
@ -90,37 +90,38 @@ Type: `object`
|
|||||||
3. _"SMALLINT"_
|
3. _"SMALLINT"_
|
||||||
4. _"INT"_
|
4. _"INT"_
|
||||||
5. _"BIGINT"_
|
5. _"BIGINT"_
|
||||||
6. _"FLOAT"_
|
6. _"BYTEINT"_
|
||||||
7. _"DOUBLE"_
|
7. _"FLOAT"_
|
||||||
8. _"DECIMAL"_
|
8. _"DOUBLE"_
|
||||||
9. _"NUMERIC"_
|
9. _"DECIMAL"_
|
||||||
10. _"TIMESTAMP"_
|
10. _"NUMERIC"_
|
||||||
11. _"TIME"_
|
11. _"TIMESTAMP"_
|
||||||
12. _"DATE"_
|
12. _"TIME"_
|
||||||
13. _"DATETIME"_
|
13. _"DATE"_
|
||||||
14. _"INTERVAL"_
|
14. _"DATETIME"_
|
||||||
15. _"STRING"_
|
15. _"INTERVAL"_
|
||||||
16. _"MEDIUMTEXT"_
|
16. _"STRING"_
|
||||||
17. _"TEXT"_
|
17. _"MEDIUMTEXT"_
|
||||||
18. _"CHAR"_
|
18. _"TEXT"_
|
||||||
19. _"VARCHAR"_
|
19. _"CHAR"_
|
||||||
20. _"BOOLEAN"_
|
20. _"VARCHAR"_
|
||||||
21. _"BINARY"_
|
21. _"BOOLEAN"_
|
||||||
22. _"VARBINARY"_
|
22. _"BINARY"_
|
||||||
23. _"ARRAY"_
|
23. _"VARBINARY"_
|
||||||
24. _"BLOB"_
|
24. _"ARRAY"_
|
||||||
25. _"LONGBLOB"_
|
25. _"BLOB"_
|
||||||
26. _"MEDIUMBLOB"_
|
26. _"LONGBLOB"_
|
||||||
27. _"MAP"_
|
27. _"MEDIUMBLOB"_
|
||||||
28. _"STRUCT"_
|
28. _"MAP"_
|
||||||
29. _"UNION"_
|
29. _"STRUCT"_
|
||||||
30. _"SET"_
|
30. _"UNION"_
|
||||||
31. _"GEOGRAPHY"_
|
31. _"SET"_
|
||||||
32. _"ENUM"_
|
32. _"GEOGRAPHY"_
|
||||||
33. _"JSON"_
|
33. _"ENUM"_
|
||||||
|
34. _"JSON"_
|
||||||
|
|
||||||
|
|
||||||
### columnConstraint
|
### constraint
|
||||||
|
|
||||||
- This enum defines the type for column constraint.
|
- This enum defines the type for column constraint.
|
||||||
- Type: `string`
|
- Type: `string`
|
||||||
@ -152,7 +153,7 @@ Type: `object`
|
|||||||
|
|
||||||
### columnName
|
### columnName
|
||||||
|
|
||||||
- Local name (not fully qualified name) of the column.
|
- Local name (not fully qualified name) of the column. ColumnName is `-` when the column is not named in struct dataType. For example, BigQuery supports struct with unnamed fields.
|
||||||
- Type: `string`
|
- Type: `string`
|
||||||
- The value must match this pattern: `^[^.]*$`
|
- The value must match this pattern: `^[^.]*$`
|
||||||
- Length: between 1 and 64
|
- Length: between 1 and 64
|
||||||
@ -168,7 +169,7 @@ Type: `object`
|
|||||||
|
|
||||||
### fullyQualifiedColumnName
|
### fullyQualifiedColumnName
|
||||||
|
|
||||||
- Fully qualified name of the column that includes `serviceName.databaseName.tableName.columnName`.
|
- Fully qualified name of the column that includes `serviceName.databaseName.tableName.columnName[.nestedColumnName]`. When columnName is null for dataType struct fields, `field_#` where `#` is field index is used. For map dataType, for key the field name `key` is used and for the value field `value` is used.
|
||||||
- Type: `string`
|
- Type: `string`
|
||||||
- Length: between 1 and 256
|
- Length: between 1 and 256
|
||||||
|
|
||||||
@ -177,12 +178,22 @@ Type: `object`
|
|||||||
|
|
||||||
- This schema defines the type for a column in a table.
|
- This schema defines the type for a column in a table.
|
||||||
- Type: `object`
|
- Type: `object`
|
||||||
|
- This schema <u>does not</u> accept additional properties.
|
||||||
- **Properties**
|
- **Properties**
|
||||||
- **name** `required`
|
- **name** `required`
|
||||||
- $ref: [#/definitions/columnName](#columnname)
|
- $ref: [#/definitions/columnName](#columnname)
|
||||||
- **columnDataType** `required`
|
- **dataType** `required`
|
||||||
- Data type of the column (int, date etc.).
|
- Data type of the column (int, date etc.).
|
||||||
- $ref: [#/definitions/columnDataType](#columndatatype)
|
- $ref: [#/definitions/dataType](#datatype)
|
||||||
|
- **arrayDataType**
|
||||||
|
- Data type used array in dataType. For example, `array<int>` has dataType as `array` and arrayDataType as `int`.
|
||||||
|
- $ref: [#/definitions/dataType](#datatype)
|
||||||
|
- **dataLength**
|
||||||
|
- Length of `char`, `varchar`, `binary`, `varbinary` `dataTypes`, else null. For example, `varchar(20)` has dataType as `varchar` and dataLength as `20`.
|
||||||
|
- Type: `integer`
|
||||||
|
- **dataTypeDisplay**
|
||||||
|
- Display name used for dataType. This is useful for complex types, such as `array<int>, map<int,string>, struct<>, and union types.
|
||||||
|
- Type: `string`
|
||||||
- **description**
|
- **description**
|
||||||
- Description of the column.
|
- Description of the column.
|
||||||
- Type: `string`
|
- Type: `string`
|
||||||
@ -193,12 +204,20 @@ Type: `object`
|
|||||||
- Type: `array`
|
- Type: `array`
|
||||||
- **Items**
|
- **Items**
|
||||||
- $ref: [../../type/tagLabel.json](../types/taglabel.md)
|
- $ref: [../../type/tagLabel.json](../types/taglabel.md)
|
||||||
- **columnConstraint**
|
- **constraint**
|
||||||
- Column level constraint.
|
- Column level constraint.
|
||||||
- $ref: [#/definitions/columnConstraint](#columnconstraint)
|
- $ref: [#/definitions/constraint](#constraint)
|
||||||
- **ordinalPosition**
|
- **ordinalPosition**
|
||||||
- Ordinal position of the column.
|
- Ordinal position of the column.
|
||||||
- Type: `integer`
|
- Type: `integer`
|
||||||
|
- **jsonSchema**
|
||||||
|
- Json schema only if the dataType is JSON else null.
|
||||||
|
- Type: `string`
|
||||||
|
- **children**
|
||||||
|
- Child columns if dataType or arrayDataType is `map`, `struct`, or `union` else `null`.
|
||||||
|
- Type: `array`
|
||||||
|
- **Items**
|
||||||
|
- $ref: [#/definitions/column](#column)
|
||||||
|
|
||||||
|
|
||||||
### columnJoins
|
### columnJoins
|
||||||
@ -278,21 +297,21 @@ Type: `object`
|
|||||||
- **nullProportion**
|
- **nullProportion**
|
||||||
- No.of null value proportion in columns.
|
- No.of null value proportion in columns.
|
||||||
- Type: `number`
|
- Type: `number`
|
||||||
- **min**
|
- **min**
|
||||||
- Minimum value in a column.
|
- Minimum value in a column.
|
||||||
- Type: `string`
|
- Type: `string`
|
||||||
- **max**
|
- **max**
|
||||||
- Maximum value in a column.
|
- Maximum value in a column.
|
||||||
- Type: `string`
|
- Type: `string`
|
||||||
- **mean**
|
- **mean**
|
||||||
- Avg value in a column.
|
- Avg value in a column.
|
||||||
- Type: `string`
|
- Type: `string`
|
||||||
- **median**
|
- **median**
|
||||||
- Median value in a column.
|
- Median value in a column.
|
||||||
- Type: `string`
|
- Type: `string`
|
||||||
- **stddev**
|
- **stddev**
|
||||||
- Standard deviation of a column.
|
- Standard deviation of a column.
|
||||||
- Type: `number`
|
- Type: `number`
|
||||||
|
|
||||||
|
|
||||||
### tableProfile
|
### tableProfile
|
||||||
@ -318,4 +337,4 @@ Type: `object`
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -49,6 +49,4 @@ This schema <u>does not</u> accept additional properties.
|
|||||||
### tag
|
### tag
|
||||||
|
|
||||||
|
|
||||||
|
_This document was updated on: Tuesday, October 12, 2021_
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
|
65
docs/openmetadata-apis/schemas/entities/task.md
Normal file
65
docs/openmetadata-apis/schemas/entities/task.md
Normal file
@ -0,0 +1,65 @@
|
|||||||
|
# Task
|
||||||
|
|
||||||
|
This schema defines the Task entity. A task is a unit of computation in a Pipeline.
|
||||||
|
|
||||||
|
**id: https://open-metadata.org/schema/entity/data/task.json**
|
||||||
|
|
||||||
|
Type: `object`
|
||||||
|
|
||||||
|
## Properties
|
||||||
|
- **id** `required`
|
||||||
|
- Unique identifier that identifies a task instance.
|
||||||
|
- $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
|
||||||
|
- **name** `required`
|
||||||
|
- Name that identifies this task instance uniquely.
|
||||||
|
- Type: `string`
|
||||||
|
- Length: between 1 and 64
|
||||||
|
- **displayName**
|
||||||
|
- Display Name that identifies this Task. It could be title or label from the pipeline services.
|
||||||
|
- Type: `string`
|
||||||
|
- **fullyQualifiedName**
|
||||||
|
- A unique name that identifies a pipeline in the format 'ServiceName.PipelineName.TaskName'.
|
||||||
|
- Type: `string`
|
||||||
|
- Length: between 1 and 64
|
||||||
|
- **description**
|
||||||
|
- Description of this Task.
|
||||||
|
- Type: `string`
|
||||||
|
- **taskUrl**
|
||||||
|
- Task URL to visit/manage. This URL points to respective pipeline service UI.
|
||||||
|
- Type: `string`
|
||||||
|
- String format must be a "uri"
|
||||||
|
- **downstreamTasks**
|
||||||
|
- All the tasks that are downstream of this task.
|
||||||
|
- Type: `array`
|
||||||
|
- **Items**
|
||||||
|
- Type: `string`
|
||||||
|
- Length: between 1 and 64
|
||||||
|
- **taskType**
|
||||||
|
- Type of the Task. Usually refers to the class it implements.
|
||||||
|
- Type: `string`
|
||||||
|
- **taskSQL**
|
||||||
|
- SQL used in the task. Can be used to determine the lineage.
|
||||||
|
- Type: `string`
|
||||||
|
- **startDate**
|
||||||
|
- Start date of the task.
|
||||||
|
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
|
||||||
|
- **endDate**
|
||||||
|
- End date of the task.
|
||||||
|
- $ref: [../../type/basic.json#/definitions/dateTime](../types/basic.md#datetime)
|
||||||
|
- **tags**
|
||||||
|
- Tags for this Pipeline.
|
||||||
|
- Type: `array`
|
||||||
|
- **Items**
|
||||||
|
- $ref: [../../type/tagLabel.json](../types/taglabel.md)
|
||||||
|
- **href**
|
||||||
|
- Link to the resource corresponding to this entity.
|
||||||
|
- $ref: [../../type/basic.json#/definitions/href](../types/basic.md#href)
|
||||||
|
- **owner**
|
||||||
|
- Owner of this pipeline.
|
||||||
|
- $ref: [../../type/entityReference.json](../types/entityreference.md)
|
||||||
|
- **service** `required`
|
||||||
|
- Link to service where this pipeline is hosted in.
|
||||||
|
- $ref: [../../type/entityReference.json](../types/entityreference.md)
|
||||||
|
|
||||||
|
|
||||||
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -43,4 +43,4 @@ Type: `object`
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -5,6 +5,7 @@ This schema defines the Thread entity. A Thread is a collection of posts made by
|
|||||||
**$id: https://open-metadata.org/schema/entity/feed/thread.json**
|
**$id: https://open-metadata.org/schema/entity/feed/thread.json**
|
||||||
|
|
||||||
Type: `object`
|
Type: `object`
|
||||||
|
|
||||||
## Properties
|
## Properties
|
||||||
- **id** `required`
|
- **id** `required`
|
||||||
- Unique identifier that identifies an entity instance.
|
- Unique identifier that identifies an entity instance.
|
||||||
@ -36,14 +37,14 @@ Type: `object`
|
|||||||
- **message** `required`
|
- **message** `required`
|
||||||
- Message in markdown format. See markdown support for more details.
|
- Message in markdown format. See markdown support for more details.
|
||||||
- Type: `string`
|
- Type: `string`
|
||||||
- **postTs**
|
- **postTs**
|
||||||
- Timestamp of the post.
|
- Timestamp of the post.
|
||||||
- Type: `string`
|
- Type: `string`
|
||||||
- String format must be a "date-time"
|
- String format must be a "date-time"
|
||||||
- **from** `required`
|
- **from** `required`
|
||||||
- ID of User (regular user or a bot) posting the message.
|
- ID of User (regular user or a bot) posting the message.
|
||||||
- $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
|
- $ref: [../../type/basic.json#/definitions/uuid](../types/basic.md#uuid)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -97,4 +97,4 @@ Type: `object`
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -59,4 +59,4 @@ This schema <u>does not</u> accept additional properties.
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -8,6 +8,8 @@ OpenMetadata supports many common types reused across different schemas. Definin
|
|||||||
|
|
||||||
{% page-ref page="dailycount.md" %}
|
{% page-ref page="dailycount.md" %}
|
||||||
|
|
||||||
|
{% page-ref page="entitylineage.md" %}
|
||||||
|
|
||||||
{% page-ref page="entityreference.md" %}
|
{% page-ref page="entityreference.md" %}
|
||||||
|
|
||||||
{% page-ref page="entityusage.md" %}
|
{% page-ref page="entityusage.md" %}
|
||||||
|
@ -34,4 +34,4 @@ Type: `object`
|
|||||||
- Type of Entity that is modified by the operation.
|
- Type of Entity that is modified by the operation.
|
||||||
- Type: `string`
|
- Type: `string`
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -84,4 +84,4 @@ This schema defines basic common types that are used by other schemas.
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -16,4 +16,4 @@ This schema <u>does not</u> accept additional properties.
|
|||||||
- **date** `required`
|
- **date** `required`
|
||||||
- $ref: [basic.json#/definitions/date](basic.md#date)
|
- $ref: [basic.json#/definitions/date](basic.md#date)
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
60
docs/openmetadata-apis/schemas/types/entitylineage.md
Normal file
60
docs/openmetadata-apis/schemas/types/entitylineage.md
Normal file
@ -0,0 +1,60 @@
|
|||||||
|
# Entity Lineage
|
||||||
|
|
||||||
|
This schema defines the type used for lineage of an entity.
|
||||||
|
|
||||||
|
**$id: https://open-metadata.org/schema/type/entitylineage.json**
|
||||||
|
|
||||||
|
Type: `object`
|
||||||
|
|
||||||
|
This schema <u>does not</u> accept additional properties.
|
||||||
|
|
||||||
|
## Properties
|
||||||
|
- **entity** `required`
|
||||||
|
- Primary entity for which this lineage graph is created.
|
||||||
|
- $ref: [entityReference.json](entityreference.md)
|
||||||
|
- **nodes**
|
||||||
|
- Type: `array`
|
||||||
|
- **Items**
|
||||||
|
- $ref: [entityReference.json](entityreference.md)
|
||||||
|
- **upstreamEdges**
|
||||||
|
- Type: `array`
|
||||||
|
- **Items**
|
||||||
|
- $ref: [#/definitions/edge](#edge)
|
||||||
|
- **downstreamEdges**
|
||||||
|
- Type: `array`
|
||||||
|
- **Items**
|
||||||
|
- $ref: [#/definitions/edge](#edge)
|
||||||
|
|
||||||
|
|
||||||
|
## Type definitions in this schema
|
||||||
|
### edge
|
||||||
|
|
||||||
|
- Edge in the lineage graph from one entity to another by entity IDs.
|
||||||
|
- Type: `object`
|
||||||
|
- **Properties**
|
||||||
|
- **fromEntity**
|
||||||
|
- From entity that is upstream of lineage edge.
|
||||||
|
- $ref: [basic.json#/definitions/uuid](basic.md#uuid)
|
||||||
|
- **toEntity**
|
||||||
|
- To entity that is downstream of lineage edge.
|
||||||
|
- $ref: [basic.json#/definitions/uuid](basic.md#uuid)
|
||||||
|
- **description**
|
||||||
|
- Type: `string`
|
||||||
|
|
||||||
|
|
||||||
|
### entitiesEdge
|
||||||
|
|
||||||
|
- Edge in the lineage graph from one entity to another using entity references.
|
||||||
|
- Type: `object`
|
||||||
|
- **Properties**
|
||||||
|
- **fromEntity**
|
||||||
|
- From entity that is upstream of lineage edge.
|
||||||
|
- $ref: [entityReference.json](entityreference.md)
|
||||||
|
- **toEntity**
|
||||||
|
- To entity that is downstream of lineage edge.
|
||||||
|
- $ref: [entityReference.json](entityreference.md)
|
||||||
|
- **description**
|
||||||
|
- Type: `string`
|
||||||
|
|
||||||
|
|
||||||
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -6,6 +6,8 @@ This schema defines the EntityReference type used for referencing an entity. Ent
|
|||||||
|
|
||||||
Type: `object`
|
Type: `object`
|
||||||
|
|
||||||
|
This schema <u>does not</u> accept additional properties.
|
||||||
|
|
||||||
## Properties
|
## Properties
|
||||||
- **id** `required`
|
- **id** `required`
|
||||||
- Unique identifier that identifies an entity instance.
|
- Unique identifier that identifies an entity instance.
|
||||||
@ -33,4 +35,4 @@ Type: `object`
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -12,8 +12,8 @@ Type: `object`
|
|||||||
- $ref: [entityReference.json](entityreference.md)
|
- $ref: [entityReference.json](entityreference.md)
|
||||||
- **usage** `required`
|
- **usage** `required`
|
||||||
- List usage details per day.
|
- List usage details per day.
|
||||||
- Type: `array`
|
- Type: `array`
|
||||||
- **Items**
|
- **Items**
|
||||||
- $ref: [usageDetails.json](usagedetails.md)
|
- $ref: [usageDetails.json](usagedetails.md)
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -30,9 +30,8 @@ Type: `object`
|
|||||||
|
|
||||||
### connectionUrl
|
### connectionUrl
|
||||||
|
|
||||||
- Type used for JDBC connection URL.
|
- Type used for JDBC connection URL of format `url_scheme://<username>:<password>@<host>:<port>/<db_name>`.
|
||||||
- Type: `string`
|
- Type: `string`
|
||||||
- String format must be a "uri"
|
|
||||||
|
|
||||||
|
|
||||||
### jdbcInfo
|
### jdbcInfo
|
||||||
@ -47,4 +46,4 @@ Type: `object`
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -17,4 +17,4 @@ Type: `object`
|
|||||||
- Total number of entries available to page through.
|
- Total number of entries available to page through.
|
||||||
- Type: `integer`
|
- Type: `integer`
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -41,4 +41,4 @@ Type: `object`
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -14,4 +14,4 @@ Type: `object`
|
|||||||
- Repeat frequency in ISO 8601 duration format. Example - 'P23DT23H'.
|
- Repeat frequency in ISO 8601 duration format. Example - 'P23DT23H'.
|
||||||
- $ref: [basic.json#/definitions/duration](basic.md#duration)
|
- $ref: [basic.json#/definitions/duration](basic.md#duration)
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -6,6 +6,8 @@ This schema defines the type for labeling an entity with a Tag.
|
|||||||
|
|
||||||
Type: `object`
|
Type: `object`
|
||||||
|
|
||||||
|
This schema <u>does not</u> accept additional properties.
|
||||||
|
|
||||||
## Properties
|
## Properties
|
||||||
- **tagFQN**
|
- **tagFQN**
|
||||||
- Type: `string`
|
- Type: `string`
|
||||||
@ -30,4 +32,4 @@ Type: `object`
|
|||||||
- Link to the tag resource.
|
- Link to the tag resource.
|
||||||
- $ref: [basic.json#/definitions/href](basic.md#href)
|
- $ref: [basic.json#/definitions/href](basic.md#href)
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
@ -39,4 +39,4 @@ Type: `object`
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
_This document was updated on: Thursday, September 16, 2021_
|
_This document was updated on: Tuesday, October 12, 2021_
|
Loading…
x
Reference in New Issue
Block a user