Keshav Mohta
103857f90c
Fixes #23010 #: BigQuery Project Selection In Profiler & AutoClassification Workflow ( #23233 )
...
* fix: added code for separate engine and session for each project in rofiler and classification and refactor billing project approach
* fix: added entity.database check, bigquery sampling tests
* fix: system metrics logic when bigquery billing project is provided
2025-09-05 14:09:14 +05:30
Teddy
29450d1104
feat: add support for DBX system metrics ( #22044 )
...
* feat: add support for DBX system metrics
* feat: add support for DBX system metrics
* fix: added WRITE back
* fix: failing test cases
* fix: failing test
2025-07-02 08:54:16 +02:00
Teddy
e4dffd281c
fix: preserve BQ struct field casing ( #21716 )
2025-06-17 23:58:35 +02:00
Mayur Singal
7760663b22
MINOR: Change ingestion licence header ( #20549 )
2025-04-03 10:39:47 +05:30
Pere Miquel Brull
c68a45e7d8
Create new Auto Classification Workflow ( #18610 )
2024-11-19 08:10:45 +01:00
Teddy
61ef55290e
MINOR - generic profiler optimization for sampling and BQ ( #14507 )
...
* fix: limit sampling to specific column
* fix: handle bigquery struct columns
* fix: default partition to 1 DAY for BQ
* fix: default to __TABLES__ for BQ table metrics
* style: ran python linting
* style: fix linting
* fix: python style
* fix: set partition to DAY if not HOUR
2023-12-27 19:13:44 +01:00
Ayush Shah
ec6184d2da
Fix Trino Dialect Import issue ( #13869 )
2023-11-07 12:10:59 +05:30
Teddy
42a426226e
Fixes Issue #11803 #12103 - Add BigQuery Struct Support ( #12435 )
...
* ref: implemented interface for profiler components + removed struct logic
* ref: ran python linting
* ref: added UML diagram to readme.md
* ref: empty commit for labeler check
* ref: remove multiple context manager for 3.7 3.8 compatibility
* ref: remove
* fix: mapper logic for BQ struct types
* feat: added BQ support for structs
* feat: clean code smell + handle null self.col.table value
* feat: ran python linting
* feat: updated test for profiler handler + disabled flaky test
* Update ingestion/tests/unit/profiler/pandas/test_sample.py
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
---------
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2023-07-14 09:12:46 +02:00
Teddy
54fbe250a1
fix: import error + BQ E2E CLI ( #12420 )
2023-07-13 13:35:37 +02:00
Teddy
b89cf64f14
Clean up profiler ( #12369 )
...
* ref: implemented interface for profiler components + removed struct logic
* ref: ran python linting
* ref: added UML diagram to readme.md
* ref: empty commit for labeler check
* ref: remove multiple context manager for 3.7 3.8 compatibility
* ref: remove
2023-07-12 17:02:32 +02:00
Hung Duong
64f147c517
Fixes #12129 : enhance bigquery sample data query ( #12130 )
...
* upgrade BigQuery Sampler
* beautify code
* revert old way of profiler & data quality, keep fetch new way sample
* Update profiler_source.py
* Update profiler_source.py
---------
Co-authored-by: hung.duong <hung.duong@be.com.vn>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
2023-07-04 08:37:15 +02:00
