Pere Miquel Brull
375e001dd9
MINOR - Fix S3 logging from ingestion pipelines ( #23590 )
...
* MINOR - Fix S3 logging from ingestion pipelines
* Update generated TypeScript types
* config
* update s3 configurations for streamable logs
* Update generated TypeScript types
* update s3 configurations for streamable logs
* update s3 configurations for streamable logs
* update s3 configurations for streamable logs
* SSE off by default
* Update log retrieval to use s3 if ingestion runner has streamable logs enabled
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Pablo Takara <pjt1991@gmail.com>
2025-10-01 09:44:17 +02:00
Mayur Singal
7760663b22
MINOR: Change ingestion licence header ( #20549 )
2025-04-03 10:39:47 +05:30
Pere Miquel Brull
d8e2187980
#15243 - Pydantic V2 & Airflow 2.9 ( #16480 )
...
* pydantic v2
* pydanticv2
* fix parser
* fix annotated
* fix model dumping
* mysql ingestion
* clean root models
* clean root models
* bump airflow
* bump airflow
* bump airflow
* optionals
* optionals
* optionals
* jdk
* airflow migrate
* fab provider
* fab provider
* fab provider
* some more fixes
* fixing tests and imports
* model_dump and model_validate
* model_dump and model_validate
* model_dump and model_validate
* union
* pylint
* pylint
* integration tests
* fix CostAnalysisReportData
* integration tests
* tests
* missing defaults
* missing defaults
2024-06-05 21:18:37 +02:00
Ayush Shah
a6ae9fd11a
Add Test Suite Implementation for Datalake ( #9235 )
2022-12-14 21:14:51 +05:30
Mayur Singal
1386b43607
Fix #6141 : Ingestion Pipeline Status Updates ( #8216 )
...
* Fix #6141 : Ingestion Pipeline Status Updates
* List Pipeline Status API & Improvements
* Rename State field to PipelineState in UI
* Convert Pipeline Status array to single object
* fix braking UI
* Rebase Fixes
* Profiler, TestStuite & DataInsigts Pipeline
* py_format
* fix logs page not loading
add pipelineStatus endpoint
* fix recent run changes
* Fix Tests
* address review comments for ui
* fix failing checks
* fix unit tests
* fix cypress test
* remove loader test as not using it
* wait for API when we click on the tab
* fix cypress waitFor api
* fix failed cypress tests
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2022-11-03 14:37:26 +05:30
Pere Miquel Brull
99f0c96df4
Fix #6278 - Lineage workflow + Usage workflow ( #6568 )
...
* Fixing conflicts
* Fixing Conflicts
* Fix snowflake
* Prepare snowflake and redshift
* snowflake use a single query
* Add bigquery
* Add clickhouse
* format
* Add query log lineage source
* Add filters to core
* Prepare mssql usage
* Update schemas
* Add example workflows
* Prepare lineage deployment
* Remove lineage
* Use filters
* Prep lineage DAG
* Prepare Lineage Ingestion
* Test lineage workflow
* format
* Comments
* fixed tests
* Revert "fixed tests"
This reverts commit 0e846c7c4806098e4adf7c071d133d1df68aedf5.
* fixed get connection method
Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-08-12 13:24:58 +05:30
