yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-09-21 15:13:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(ingest): fix bigquery-usage regex for both partitioned and sharded tables (#3571)

Browse Source
This commit is contained in:
Swaroop Jagadish 2021-11-13 19:43:59 -08:00 committed by GitHub
parent d33d9cbb17
commit 482dac6d2e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
metadata-ingestion/src/datahub/ingestion/source/usage/bigquery_usage.py
View File

@ -37,7 +37,8 @@ GCP_LOGGING_PAGE_SIZE = 1000
# Handle yearly, monthly, daily, or hourly partitioning.
# See https://cloud.google.com/bigquery/docs/partitioned-tables.
PARTITIONED_TABLE_REGEX = re.compile(r"^(.+)\$(\d{4}|\d{6}|\d{8}|\d{10})$")
# This REGEX handles both Partitioned Tables ($ separator) and Sharded Tables (_ separator)
PARTITIONED_TABLE_REGEX = re.compile(r"^(.+)[\$_](\d{4}|\d{6}|\d{8}|\d{10})$")
# Handle table snapshots
# See https://cloud.google.com/bigquery/docs/table-snapshots-intro.