yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-11-03 20:27:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
datahub/metadata-ingestion/examples/structured_properties/README.md
david-leifker 943bb57cbc
feat(backend): structured properties and forms (#9626) 
Co-authored-by: Chris Collins <chriscollins3456@gmail.com>
Co-authored-by: RyanHolstien <RyanHolstien@users.noreply.github.com>
2024-01-22 11:46:04 -06:00

1.4 KiB
Raw Blame History

Extended Properties

Expected Capabilities

structured_properties command

- id: io.acryl.privacy.retentionTime
  # urn: urn:li:structuredProperty:<>
  # fullyQualifiedName: io.acryl.privacy.retentionTime
  type: STRING
  cardinality: MULTIPLE
  entityTypes:
    - dataset # or urn:li:logicalEntity:metamodel.datahub.dataset
    - dataflow
  description: "Retention Time is used to figure out how long to retain records in a dataset"
  allowedValues:
    - value: 30 days
      description: 30 days, usually reserved for datasets that are ephemeral and contain pii
    - value: 3 months
      description: Use this for datasets that drive monthly reporting but contain pii
    - value: 2 yrs
      description: Use this for non-sensitive data that can be retained for longer
- id: io.acryl.dataManagement.replicationSLA
  type: NUMBER
  description: "SLA for how long data can be delayed before replicating to the destination cluster"
  entityTypes:
    - dataset
- id: io.acryl.dataManagement.deprecationDate
  type: DATE
  entityTypes:
    - dataset
    - dataFlow
    - dataJob

datahub properties create -f structured_properties.yaml

datahub properties create --name io.acryl.privacy.retentionTime --type STRING --cardinality MULTIPLE --entity_type DATASET --entity_type DATAFLOW

dataset command

datahub dataset create -f dataset.yaml

See example in dataproduct.