yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-11-03 12:08:31 +00:00
Code Issues Packages Projects Releases Wiki Activity

MINOR: Ignore Cluster Information from columns (#16495)

Browse Source
This commit is contained in:
Mayur Singal 2024-05-31 15:41:12 +05:30 committed by GitHub
parent 136ec635a7
commit 22be0edd79
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 5 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
ingestion/src/metadata/ingestion/source/database/databricks/metadata.py
View File

@ -135,7 +135,11 @@ def get_columns(self, connection, table_name, schema=None, **kw):
result = []
for col_name, col_type, _comment in rows:
# Handle both oss hive and Databricks' hive partition header, respectively
if col_name in ("# Partition Information", "# Partitioning"):
if col_name in (
"# Partition Information",
"# Partitioning",
"# Clustering Information",
):
break
# Take out the more detailed type information
# e.g. 'map<ixnt,int>' -> 'map'