Pere Miquel Brull
c68a45e7d8
Create new Auto Classification Workflow ( #18610 )
2024-11-19 08:10:45 +01:00
Teddy
61ef55290e
MINOR - generic profiler optimization for sampling and BQ ( #14507 )
...
* fix: limit sampling to specific column
* fix: handle bigquery struct columns
* fix: default partition to 1 DAY for BQ
* fix: default to __TABLES__ for BQ table metrics
* style: ran python linting
* style: fix linting
* fix: python style
* fix: set partition to DAY if not HOUR
2023-12-27 19:13:44 +01:00
Ayush Shah
ec6184d2da
Fix Trino Dialect Import issue ( #13869 )
2023-11-07 12:10:59 +05:30
Teddy
42a426226e
Fixes Issue #11803 #12103 - Add BigQuery Struct Support ( #12435 )
...
* ref: implemented interface for profiler components + removed struct logic
* ref: ran python linting
* ref: added UML diagram to readme.md
* ref: empty commit for labeler check
* ref: remove multiple context manager for 3.7 3.8 compatibility
* ref: remove
* fix: mapper logic for BQ struct types
* feat: added BQ support for structs
* feat: clean code smell + handle null self.col.table value
* feat: ran python linting
* feat: updated test for profiler handler + disabled flaky test
* Update ingestion/tests/unit/profiler/pandas/test_sample.py
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
---------
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2023-07-14 09:12:46 +02:00
Teddy
54fbe250a1
fix: import error + BQ E2E CLI ( #12420 )
2023-07-13 13:35:37 +02:00
Teddy
b89cf64f14
Clean up profiler ( #12369 )
...
* ref: implemented interface for profiler components + removed struct logic
* ref: ran python linting
* ref: added UML diagram to readme.md
* ref: empty commit for labeler check
* ref: remove multiple context manager for 3.7 3.8 compatibility
* ref: remove
2023-07-12 17:02:32 +02:00
Hung Duong
64f147c517
Fixes #12129 : enhance bigquery sample data query ( #12130 )
...
* upgrade BigQuery Sampler
* beautify code
* revert old way of profiler & data quality, keep fetch new way sample
* Update profiler_source.py
* Update profiler_source.py
---------
Co-authored-by: hung.duong <hung.duong@be.com.vn>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
2023-07-04 08:37:15 +02:00
