14 Commits

Author SHA1 Message Date
Teddy
29450d1104
feat: add support for DBX system metrics (#22044)
* feat: add support for DBX system metrics

* feat: add support for DBX system metrics

* fix: added WRITE back

* fix: failing test cases

* fix: failing test
2025-07-02 08:54:16 +02:00
Mayur Singal
7760663b22
MINOR: Change ingestion licence header (#20549) 2025-04-03 10:39:47 +05:30
Imri Paran
68e71cb3dc
GEN-970: Refactor redshift system metrics to support freshness test (#17981)
* ref(profiler): redshift system metrics

- moved redshift system metrics to the redshift source module
- use Timestamp in data quality
- added plugin feature to test utils

* use timezone.utc

* format

* reverted unintended snowflake changes

* fixed import test_system_metrics.py

* revert

* fixed import in tests
2024-10-10 08:32:07 +02:00
Imri Paran
84be1a3162
Fix 17698: use resolution logic for snowflake system metrics profiler (#17699)
* fix(profiler): snowflake

resolve tables using the snowflake engine instead of OpenMetadata

* added env for cleaning up dbs in E2E

* moved system metric method to profiler. all the rest says in snowflake

* format

* revert unnecessary changes

* removed test for previous resolution method

* use shutdown39
2024-09-06 07:25:10 +00:00
Pere Miquel Brull
d8e2187980
#15243 - Pydantic V2 & Airflow 2.9 (#16480)
* pydantic v2

* pydanticv2

* fix parser

* fix annotated

* fix model dumping

* mysql ingestion

* clean root models

* clean root models

* bump airflow

* bump airflow

* bump airflow

* optionals

* optionals

* optionals

* jdk

* airflow migrate

* fab provider

* fab provider

* fab provider

* some more fixes

* fixing tests and imports

* model_dump and model_validate

* model_dump and model_validate

* model_dump and model_validate

* union

* pylint

* pylint

* integration tests

* fix CostAnalysisReportData

* integration tests

* tests

* missing defaults

* missing defaults
2024-06-05 21:18:37 +02:00
Pere Miquel Brull
db985fda57
MINOR - Snowflake system queries to work with ES & IDENTIFIER (#14864) 2024-01-26 18:41:16 +05:30
Teddy
d228a93fbf
fix: increase floating point precision (#14827) 2024-01-24 09:19:19 +01:00
Teddy
10904049e4
fix: handle lower and upper case name (#13778) 2023-10-31 09:51:13 +01:00
Ayush Shah
ab1ec50c2c
Fixes Mssql Ntext, text and Image (#12490) 2023-07-20 13:34:35 +05:30
Teddy
b89cf64f14
Clean up profiler (#12369)
* ref: implemented interface for profiler components + removed struct logic

* ref: ran python linting

* ref: added UML diagram to readme.md

* ref: empty commit for labeler check

* ref: remove multiple context manager for 3.7 3.8 compatibility

* ref: remove
2023-07-12 17:02:32 +02:00
Teddy
1e86b6533c
Fixes #11743 - Remove SQLParse dependency for System Metrics (#12072)
* fix: removed sqlparse dependency for system metrics

* fix: update sample query

* fix: move system test os retrieval to `.get()`

* fix: move os.environ to `get`
2023-06-22 06:51:24 +02:00
Teddy
60de33d7cf
Fixes #11384 - Implement mem. optimization for sys. metrics (#11460)
* fix: optimize system metrics retrieval for memory

* fix: ran python linting

* fix: logic to retrieve unique system metrics operations

* fix: added logic to clean up query before parsing it

* fix: added E2E tests for rds, bq, snflk system metrics

* fix: ran python linting

* fix: fix postgres query + add default byte size to env var

* fix: ran python linting
2023-05-09 12:05:35 +02:00
Teddy
a7d98dddda
Fixes #9632 - Add Profiler Support for BQ Arrays of Structs (#11059) 2023-04-14 19:29:26 +05:30
Teddy
fc9b64b52a
Fix histogram labels (#10797)
* fix: added logic to format bin labels for histogram metric

* fix: histogram labels

* fix: added types for arguments
2023-03-28 12:59:45 +02:00