IceS2
e79c54e6a5
MINOR: Add injection to profiler ( #21738 )
...
* Initial implementation for our Connection Class
* Implement the Initial Connection class
* Add Unit Tests
* Implement Dependency Injection for the Ingestion Framework
* Fix Test
* Fix Profile Test Connection
* Add Injection to Metrics in Profiler
* Add Injection to the Profiler
* Fix UnitTests
* Fix Pytests
* Fix Tests
* Fix types
2025-06-17 19:01:00 +02:00
Mayur Singal
7760663b22
MINOR: Change ingestion licence header ( #20549 )
2025-04-03 10:39:47 +05:30
Teddy
58699063db
MINOR -- Fix DQ Partition Issue ( #18641 )
...
* fix: renamed `random_sample` to `get_dataset` and change dunder method access for SQA Table object
* fix: removed handle_partition decorator
* fix: fixed DQ partition issue + moved to `tablesample` method
* style: ran python linting
* style: fix python format check issues
* feat: added postgres tablesample
* style: ran python linting
* fix: sampling delta
* fix: merge conflicts
* fix: resolved conflicts
* style: ran python linting
* fix: patch orm call in test case
* fix: mock build_table_orm call in tests
* fix: test case failures and errors
* fix: removed unused import
* fix: patch typo
* fix: trino table schema retrieval
* fix: remove tuple context manager for 3.8 test support
2024-11-27 08:50:54 +01:00
Pere Miquel Brull
c68a45e7d8
Create new Auto Classification Workflow ( #18610 )
2024-11-19 08:10:45 +01:00
IceS2
f0049853ec
FIXES 14885: Initial deltalake implementation for s3 ( #16665 )
...
* Initial deltalake implementation for s3
* Fix styles
* Fix test_amundsen
* Fix UnitTests
* Fix Checkstyle
* Fix integration tests due to datalake client refactor
* Fix unit tests
* Fix tests
* Fix Integration DeltaLake Storage test
* Skip delta storage integration test for python 3.8
* DeltaLake JSONSchema changes migrations
* Update import name
* Add some comments based on sonarcloud suggestions
* Update DeltaLake documentation
* Resolve some comments
2024-06-20 12:08:21 +05:30
Mayur Singal
7359d6210c
MINOR: Fix Profiler for SSL Enabled Source ( #16613 )
2024-06-12 11:40:30 +05:30
Pere Miquel Brull
cb72a22b59
Fix - e2e tests for pydantic V2 ( #16551 )
...
* Fix - e2e tests for pydantic V2
* add correct default
* add correct default
* revert datetime aware
* revert datetime aware
* revert datetime aware
* revert datetime aware
* revert datetime aware
* revert datetime aware
* revert datetime aware
* revert datetime aware
* fix apis
* format
2024-06-06 19:36:17 -07:00
Pere Miquel Brull
d8e2187980
#15243 - Pydantic V2 & Airflow 2.9 ( #16480 )
...
* pydantic v2
* pydanticv2
* fix parser
* fix annotated
* fix model dumping
* mysql ingestion
* clean root models
* clean root models
* bump airflow
* bump airflow
* bump airflow
* optionals
* optionals
* optionals
* jdk
* airflow migrate
* fab provider
* fab provider
* fab provider
* some more fixes
* fixing tests and imports
* model_dump and model_validate
* model_dump and model_validate
* model_dump and model_validate
* union
* pylint
* pylint
* integration tests
* fix CostAnalysisReportData
* integration tests
* tests
* missing defaults
* missing defaults
2024-06-05 21:18:37 +02:00
Ayush Shah
b79e5c064b
Fix 15576 - Eval Data Type issue fix ( #15702 )
2024-04-03 15:51:19 +05:30
Teddy
9a4a9df836
Fix #14895 - Get Metadata from Parquet Schema ( #14956 )
...
* linting: fix python linting
* fix: get column types from parquet schema for parquet files
* style: python linting
* fix: remove displayType check in test as variation depending on OS
2024-02-01 09:02:52 +01:00
Teddy
c7ac28f2c2
Fixes #11357 - Implement profiler custom metric processing ( #14021 )
...
* feat: add backend support for custom metrics
* feat: fix python test
* feat: support custom metrics computation
* feat: updated tests for custom metrics
* feat: added dl support for min max of datetime
* feat: added is safe query check for query sampler
* feat: added support for custom metric computation in dl
* feat: added explicit addProper for pydantic model import fo Extra
* feat: added custom metric to returned obj
* feat: wrapped trino import in __init__
* feat: fix python linting
* feat: fix typing in 3.8
2023-11-17 17:51:39 +01:00
