9 Commits

Author	SHA1	Message	Date
Teddy	a853561d30	MINOR: data sample ingestion bigquery (#21074) ... * fix: data sample ingestion bigquery * style: ran python linting * fix: flaky test in topology	2025-05-06 15:58:37 +02:00
Mayur Singal	7760663b22	MINOR: Change ingestion licence header (#20549)	2025-04-03 10:39:47 +05:30
Pere Miquel Brull	d8e2187980	#15243 - Pydantic V2 & Airflow 2.9 (#16480) ... * pydantic v2 * pydanticv2 * fix parser * fix annotated * fix model dumping * mysql ingestion * clean root models * clean root models * bump airflow * bump airflow * bump airflow * optionals * optionals * optionals * jdk * airflow migrate * fab provider * fab provider * fab provider * some more fixes * fixing tests and imports * model_dump and model_validate * model_dump and model_validate * model_dump and model_validate * union * pylint * pylint * integration tests * fix CostAnalysisReportData * integration tests * tests * missing defaults * missing defaults	2024-06-05 21:18:37 +02:00
IceS2	e7c9d6aa7f	FIXES 15215: Implement initial Multithreading approach for the Metadata Ingestion on Databases (#15130) ... * Implement Initial MultiThread suggestion * Update all the ingestion sources to use the new ContextManager * Fix missing wraps on decorator * Fix Unittests * Fix linters * Fix linters * Fix BigQuery UnitTests * Add UnitTests to the newly created code * Fix unittest * change the threads from table to schemas * Update README.md * Small change suggested by Sonar * Slight change to test a different way to multithread over tables * Debug changes * More multithread tests * Remove uneeded wait time * Testing * refactor code based on removal of time.sleep * Fix wrong paste * Improve ExecutionTimeContextManager * Fix missing .get() and unit tests * Fix conflicting changes * Update Multithread logic with the incremental extraction * Fix linters * Fix unittest * Remove commented code * Fix Unittests * Fix checkstyle * Change default to threads = 1	2024-03-25 18:20:40 +01:00
Pere Miquel Brull	85e2058979	MINOR - Fix & Organize topology context (#14838) ... * MINOR - Fix & Organize topology context * Handle missing context charts	2024-01-25 08:22:07 +01:00
Pere Miquel Brull	f4bbca3f72	MINOR - Clean topology & add tests (#14527) ... * Clean topo * Format * Add tests * Fix tests * Merge main	2023-12-29 17:00:59 +01:00
Pere Miquel Brull	442528267c	Simplify topology & update context management (#13196)	2023-09-15 09:44:42 +02:00
Pere Miquel Brull	6c0e9f5061	Part of #7272 - Centralize Workflows, Status, and Exception Management (#13029) ... * Prep changes * Prep changes * prep changes * Update imports * Format * Prep delete * Prep delete * Fix sink * Prep test * Commit * passing either * passing either * Prep Either * Metadata source with Either * Update status * Merge remote-tracking branch 'upstream/main' into issue-7272 * Format * Linting * Linting * Linting * Linting * Fix tests * Fix tests * Fix tests * Fix tests * Fix tests * Fix tests * Fix tests * Comments	2023-08-30 15:49:42 +02:00
Pere Miquel Brull	02fece727b	Fix #5307 - Define a better topology for Ingestion Sources (#5366) ... Fix #5307 - Define a better topology for Ingestion Sources (#5366)	2022-06-13 12:17:54 +02:00