yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-08-23 00:28:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(docs): formatting of transformers code blocks (#10670)

Browse Source
This commit is contained in:
Alex Waldron 2024-06-15 01:02:34 +01:00 committed by GitHub
parent e66726b9cc
commit edb9cf61f7
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 80 additions and 85 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
metadata-ingestion/docs/transformer/dataset_transformer.md
View File

@ -817,8 +817,6 @@ overwrite the previous value.
properties: properties:
prop1: value1 prop1: value1
prop2: value2 prop2: value2
``` ```
- Add dataset-properties, however overwrite the dataset-properties available for the dataset on DataHub GMS - Add dataset-properties, however overwrite the dataset-properties available for the dataset on DataHub GMS
```yaml ```yaml
@ -829,8 +827,6 @@ overwrite the previous value.
properties: properties:
prop1: value1 prop1: value1
prop2: value2 prop2: value2
``` ```
- Add dataset-properties, however keep the dataset-properties available for the dataset on DataHub GMS - Add dataset-properties, however keep the dataset-properties available for the dataset on DataHub GMS
```yaml ```yaml
@ -841,7 +837,6 @@ overwrite the previous value.
properties: properties:
prop1: value1 prop1: value1
prop2: value2 prop2: value2
``` ```
## Add Dataset datasetProperties ## Add Dataset datasetProperties
@ -973,7 +968,7 @@ transformers:
`simple_add_dataset_domain` can be configured in below different way `simple_add_dataset_domain` can be configured in below different way
- Add domains, however replace existing domains sent by ingestion source - Add domains, however replace existing domains sent by ingestion source
```yaml ```yaml
transformers: transformers:
- type: "simple_add_dataset_domain" - type: "simple_add_dataset_domain"
config: config:
@ -983,7 +978,7 @@ transformers:
- "urn:li:domain:hr" - "urn:li:domain:hr"
``` ```
- Add domains, however overwrite the domains available for the dataset on DataHub GMS - Add domains, however overwrite the domains available for the dataset on DataHub GMS
```yaml ```yaml
transformers: transformers:
- type: "simple_add_dataset_domain" - type: "simple_add_dataset_domain"
config: config:
@ -993,7 +988,7 @@ transformers:
- "urn:li:domain:hr" - "urn:li:domain:hr"
``` ```
- Add domains, however keep the domains available for the dataset on DataHub GMS - Add domains, however keep the domains available for the dataset on DataHub GMS
```yaml ```yaml
transformers: transformers:
- type: "simple_add_dataset_domain" - type: "simple_add_dataset_domain"
config: config:
@ -1032,7 +1027,7 @@ in both of the cases domain should be provisioned on DataHub GMS
`pattern_add_dataset_domain` can be configured in below different way `pattern_add_dataset_domain` can be configured in below different way
- Add domains, however replace existing domains sent by ingestion source - Add domains, however replace existing domains sent by ingestion source
```yaml ```yaml
transformers: transformers:
- type: "pattern_add_dataset_domain" - type: "pattern_add_dataset_domain"
config: config:
@ -1043,7 +1038,7 @@ in both of the cases domain should be provisioned on DataHub GMS
'urn:li:dataset:\(urn:li:dataPlatform:postgres,postgres\.public\.t.*': ["urn:li:domain:finance"] 'urn:li:dataset:\(urn:li:dataPlatform:postgres,postgres\.public\.t.*': ["urn:li:domain:finance"]
``` ```
- Add domains, however overwrite the domains available for the dataset on DataHub GMS - Add domains, however overwrite the domains available for the dataset on DataHub GMS
```yaml ```yaml
transformers: transformers:
- type: "pattern_add_dataset_domain" - type: "pattern_add_dataset_domain"
config: config:
@ -1054,7 +1049,7 @@ in both of the cases domain should be provisioned on DataHub GMS
'urn:li:dataset:\(urn:li:dataPlatform:postgres,postgres\.public\.t.*': ["urn:li:domain:finance"] 'urn:li:dataset:\(urn:li:dataPlatform:postgres,postgres\.public\.t.*': ["urn:li:domain:finance"]
``` ```
- Add domains, however keep the domains available for the dataset on DataHub GMS - Add domains, however keep the domains available for the dataset on DataHub GMS
```yaml ```yaml
transformers: transformers:
- type: "pattern_add_dataset_domain" - type: "pattern_add_dataset_domain"
config: config:
@ -1099,7 +1094,7 @@ transformers:
`domain_mapping_based_on_tags` can be configured in below different way `domain_mapping_based_on_tags` can be configured in below different way
- Add domains based on tags, however overwrite the domains available for the dataset on DataHub GMS - Add domains based on tags, however overwrite the domains available for the dataset on DataHub GMS
```yaml ```yaml
transformers: transformers:
- type: "domain_mapping_based_on_tags" - type: "domain_mapping_based_on_tags"
config: config:
@ -1109,7 +1104,7 @@ transformers:
'example2': "urn:li:domain:hr" 'example2': "urn:li:domain:hr"
``` ```
- Add domains based on tags, however keep the domains available for the dataset on DataHub GMS - Add domains based on tags, however keep the domains available for the dataset on DataHub GMS
```yaml ```yaml
transformers: transformers:
- type: "domain_mapping_based_on_tags" - type: "domain_mapping_based_on_tags"
config: config:
@ -1323,7 +1318,7 @@ Next we need to tell the helper classes which entity types and aspect we are int
def entity_types(self) -> List[str]: def entity_types(self) -> List[str]:
return ["dataset"] return ["dataset"]
def aspect_name(self) -> str: def aspect_name(self) -> str:
return "ownership" return "ownership"
``` ```
@ -1332,9 +1327,9 @@ Finally we need to implement the `transform_aspect()` method that does the work
```python ```python
# add this as a function of AddCustomOwnership # add this as a function of AddCustomOwnership
def transform_aspect( # type: ignore def transform_aspect( # type: ignore
self, entity_urn: str, aspect_name: str, aspect: Optional[OwnershipClass] self, entity_urn: str, aspect_name: str, aspect: Optional[OwnershipClass]
) -> Optional[OwnershipClass]: ) -> Optional[OwnershipClass]:
owners_to_add = self.owners owners_to_add = self.owners
assert aspect is None or isinstance(aspect, OwnershipClass) assert aspect is None or isinstance(aspect, OwnershipClass)