yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-08-18 14:06:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fixes GEN-2199: Allow Fivetran filtering of pipelines using name instead of id (#18929)

Browse Source
This commit is contained in:
Ayush Shah 2024-12-05 10:55:11 +05:30 committed by GitHub
parent 8ac80e6807
commit 8664c8df75
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
3 changed files with 7 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
ingestion/src/metadata/ingestion/source/pipeline/fivetran/metadata.py
View File

@ -275,7 +275,7 @@ class FivetranSource(PipelineServiceSource):
)
def get_pipeline_name(self, pipeline_details: FivetranPipelineDetails) -> str:
return pipeline_details.pipeline_name
return pipeline_details.pipeline_display_name or pipeline_details.pipeline_name
def get_source_url(
self,

2
ingestion/tests/unit/topology/pipeline/test_fivetran.py
View File

@ -145,7 +145,7 @@ class FivetranUnitTest(TestCase):
def test_pipeline_name(self):
assert (
self.fivetran.get_pipeline_name(EXPECTED_FIVETRAN_DETAILS)
== f'{mock_data.get("group").get("id")}_{mock_data.get("source").get("id")}'
== f'{mock_data.get("group").get("name")} <> {mock_data.get("source").get("schema")}'
)
def test_pipelines(self):

5
openmetadata-docs/content/partials/v1.6/deployment/upgrade/upgrade-prerequisites.md
View File

@ -134,3 +134,8 @@ The `ServiceSpec` class serves as the entrypoint for the connector and holds the
to ingest and process the metadata from the source.
You can see [postgres](https://github.com/open-metadata/OpenMetadata/blob/main/ingestion/src/metadata/ingestion/source/database/postgres/service_spec.py) for an
implementation example.
### Fivetran
The filtering of Fivetran pipelines now supports using their names instead of IDs. This change may affect existing configurations that rely on pipeline IDs for filtering.