mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-07-07 17:18:23 +00:00

* updated schema documentaiton

* id correction

2021-11-15 08:05:54 -08:00

12 KiB

Raw Permalink Blame History

Table

This schema defines the Table entity. A Table organizes data in rows and columns and is defined by a Schema. OpenMetadata does not have a separate abstraction for Schema. Both Table and Schema are captured in this entity.

$id:https://open-metadata.org/schema/entity/data/table.json

Type: object

This schema does not accept additional properties.

Properties

id required
- Unique identifier of this table instance.
- $ref: ../../type/basic.json#/definitions/uuid
name required
- Name of a table. Expected to be unique within a database.
- $ref: #/definitions/tableName
displayName
- Display Name that identifies this table. It could be title or label from the source services.
- Type: string
fullyQualifiedName
- Fully qualified name of a table in the form serviceName.databaseName.tableName.
- Type: string
description
- Description of a table.
- Type: string
version
- Metadata version of the entity.
- $ref: ../../type/entityHistory.json#/definitions/entityVersion
updatedAt
- Last update time corresponding to the new version of the entity.
- $ref: ../../type/basic.json#/definitions/dateTime
updatedBy
- User who made the update.
- Type: string
href
- Link to this table resource.
- $ref: ../../type/basic.json#/definitions/href
tableType
- $ref: #/definitions/tableType
columns required
- Columns in this table.
- Type: array
  - Items
  - $ref: #/definitions/column
tableConstraints
- Table constraints.
- Type: array
  - Items
  - $ref: #/definitions/tableConstraint
owner
- Owner of this table.
- $ref: ../../type/entityReference.json
database
- Reference to Database that contains this table.
- $ref: ../../type/entityReference.json
location
- Reference to the Location that contains this table.
- $ref: ../../type/entityReference.json
viewDefinition
- View Definition in SQL. Applies to TableType.View only.
- $ref: ../../type/basic.json#/definitions/sqlQuery
tags
- Tags for this table.
- Type: array
  - Items
  - $ref: ../../type/tagLabel.json
usageSummary
- Latest usage information for this table.
- $ref: ../../type/usageDetails.json
followers
- Followers of this table.
- $ref: ../../type/entityReference.json#/definitions/entityReferenceList
joins
- Details of other tables this table is frequently joined with.
- $ref: #/definitions/tableJoins
sampleData
- Sample data for a table.
- $ref: #/definitions/tableData
tableProfile
- Data profile for a table.
- Type: array
  - Items
  - $ref: #/definitions/tableProfile
tableQueries
- List of queries that ran against a table.
- Type: array
  - Items
  - $ref: #/definitions/sqlQuery
changeDescription
- Change that lead to this version of the entity.
- $ref: ../../type/entityHistory.json#/definitions/changeDescription

Type definitions in this schema

tableType

This schema defines the type used for describing different types of tables.
Type: string
The value is restricted to the following:
1. "Regular"
2. "External"
3. "View"
4. "SecureView"
5. "MaterializedView"

dataType

This enum defines the type of data stored in a column.
Type: string
The value is restricted to the following:
1. "NUMBER"
2. "TINYINT"
3. "SMALLINT"
4. "INT"
5. "BIGINT"
6. "BYTEINT"
7. "FLOAT"
8. "DOUBLE"
9. "DECIMAL"
10. "NUMERIC"
11. "TIMESTAMP"
12. "TIME"
13. "DATE"
14. "DATETIME"
15. "INTERVAL"
16. "STRING"
17. "MEDIUMTEXT"
18. "TEXT"
19. "CHAR"
20. "VARCHAR"
21. "BOOLEAN"
22. "BINARY"
23. "VARBINARY"
24. "ARRAY"
25. "BLOB"
26. "LONGBLOB"
27. "MEDIUMBLOB"
28. "MAP"
29. "STRUCT"
30. "UNION"
31. "SET"
32. "GEOGRAPHY"
33. "ENUM"
34. "JSON"

constraint

This enum defines the type for column constraint.
Type: string
The value is restricted to the following:
1. "NULL"
2. "NOT_NULL"
3. "UNIQUE"
4. "PRIMARY_KEY"

tableConstraint

This enum defines the type for table constraint.
Type: object
Properties
- constraintType
  - Type: string
  - The value is restricted to the following:
    1. "UNIQUE"
    2. "PRIMARY_KEY"
    3. "FOREIGN_KEY"
- columns
  - List of column names corresponding to the constraint.
  - Type: array
    - Items
    - Type: string

columnName

Local name (not fully qualified name) of the column. ColumnName is - when the column is not named in struct dataType. For example, BigQuery supports struct with unnamed fields.
Type: string
The value must match this pattern: ^[^.]*$
Length: between 1 and 64

tableName

Local name (not fully qualified name) of a table.
Type: string
The value must match this pattern: ^[^.]*$
Length: between 1 and 64

fullyQualifiedColumnName

Fully qualified name of the column that includes serviceName.databaseName.tableName.columnName[.nestedColumnName]. When columnName is null for dataType struct fields, field_# where # is field index is used. For map dataType, for key the field name key is used and for the value field value is used.
Type: string
Length: between 1 and 256

column

This schema defines the type for a column in a table.
Type: object
This schema does not accept additional properties.
Properties
- name required
  - $ref: #/definitions/columnName
- dataType required
  - Data type of the column (int, date etc.).
  - $ref: #/definitions/dataType
- arrayDataType
  - Data type used array in dataType. For example, array<int> has dataType as array and arrayDataType as int.
  - $ref: #/definitions/dataType
- dataLength
  - Length of char, varchar, binary, varbinary dataTypes, else null. For example, varchar(20) has dataType as varchar and dataLength as 20.
  - Type: integer
- dataTypeDisplay
  - Display name used for dataType. This is useful for complex types, such as `array, map<int,string>, struct<>, and union types.
  - Type: string
- description
  - Description of the column.
  - Type: string
- fullyQualifiedName
  - $ref: #/definitions/fullyQualifiedColumnName
- tags
  - Tags associated with the column.
  - Type: array
    - Items
    - $ref: ../../type/tagLabel.json
- constraint
  - Column level constraint.
  - $ref: #/definitions/constraint
- ordinalPosition
  - Ordinal position of the column.
  - Type: integer
- jsonSchema
  - Json schema only if the dataType is JSON else null.
  - Type: string
- children
  - Child columns if dataType or arrayDataType is map, struct, or union else null.
  - Type: array
    - Items
    - $ref: #/definitions/column

columnJoins

This schema defines the type to capture how frequently a column is joined with columns in the other tables.
Type: object
This schema does not accept additional properties.
Properties
- columnName
  - $ref: #/definitions/columnName
- joinedWith
  - Fully qualified names of the columns that this column is joined with.
  - Type: array
    - Items
      - Type: object
    - Properties
      - fullyQualifiedName
        
        $ref: #/definitions/fullyQualifiedColumnName
      - joinCount
        
        Type: integer

tableJoins

This schema defines the type to capture information about how columns in this table are joined with columns in the other tables.
Type: object
This schema does not accept additional properties.
Properties
- startDate
  - Date can be only from today going back to last 29 days.
  - $ref: ../../type/basic.json#/definitions/date
- dayCount
  - Type: integer
  - Default: 1
- columnJoins
  - Type: array
    - Items
    - $ref: #/definitions/columnJoins

tableData

This schema defines the type to capture rows of sample data for a table.
Type: object
This schema does not accept additional properties.
Properties
- columns
  - List of local column names (not fully qualified column names) of the table.
  - Type: array
    - Items
    - $ref: #/definitions/columnName
  - rows
    - Data for multiple rows of the table.
    - Type: array
      - Items
        
        Data for a single row of the table within the same order as columns fields.
        
        Type: array

columnProfile

This schema defines the type to capture the table's column profile.
Type: object
This schema does not accept additional properties.
Properties
- name
  - Column Name.
  - Type: string
- uniqueCount
  - No. of unique values in the column.
  - Type: number
- uniqueProportion
  - Proportion of number of unique values in a column.
  - Type: number
- nullCount
  - No.of null values in a column.
  - Type: number
- nullProportion
  - No.of null value proportion in columns.
  - Type: number
- min
  - Minimum value in a column.
  - Type: string
- max
  - Maximum value in a column.
  - Type: string
- mean
  - Avg value in a column.
  - Type: string
- median
  - Median value in a column.
  - Type: string
- stddev
  - Standard deviation of a column.
  - Type: number

tableProfile

This schema defines the type to capture the table's data profile.
Type: object
This schema does not accept additional properties.
Properties
- profileDate
  - Data one which profile is taken.
  - $ref: ../../type/basic.json#/definitions/date
- columnCount
  - No.of columns in the table.
  - Type: number
- rowCount
  - No.of rows in the table.
  - Type: number
- columnProfile
  - List of local column profiles of the table.
  - Type: array
    - Items
    - $ref: #/definitions/columnProfile

sqlQuery

This schema defines the type to capture the table's sql queries.
Type: object
Properties
- query
  - SQL Query text that matches the table name.
  - Type: string
- duration
  - How long did the query took to run in seconds.
  - Type: number
- user
  - User who ran this query.
  - $ref: ../../type/entityReference.json
- vote
  - Users can vote up to rank the popular queries.
  - Type: number
  - Default: 1
- checksum
  - Checksum to avoid registering duplicate queries.
  - Type: string
- queryDate
  - Date on which the query ran.
  - $ref: ../../type/basic.json#/definitions/date

This document was updated on: Monday, November 15, 2021

12 KiB Raw Permalink Blame History