Mayuri Nehate 223650dd7a
feat(ingest): add bigquery-queries source (#10994)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-25 22:51:00 -07:00

22 lines
595 B
YAML

source:
type: bigquery
config:
dataset_pattern:
allow:
- finance_bq_dataset
table_pattern:
deny:
# The exact name of the table is revenue_table_name
# The reason we have this `.*` at the beginning is because the current implmenetation of table_pattern is testing
# project_id.dataset_name.table_name
# We will improve this in the future
- .*revenue_table_name
include_table_lineage: true
include_usage_statistics: true
profiling:
enabled: true
profile_table_level_only: true
sink:
# sink configs