Nahuel
a878aa911c
Fix#6212: Retrieve connection params from secret manager in CLI commands ( #6441 )
...
* Retrieve connection params from secret manager for database connectors
* Retrieve connection params from secret manager for all services except database connectors
* Stop retrieving connection from SM in Airflow rest plugin
* Retrieve connection params from secret manager for dashboard services
* Retrieve connection params when initializing Workflow/ProfilerWorkflow objects
* Align services topologies + comment changes in topology runner
* Address SonarCloud bug detected
* Update database service topology
* Address PR comments
* Address PR comments
* Address PR comments
2022-08-02 09:13:46 +02:00
Mayur Singal
bafca3b7b6
Fix #6387 : Clean Ingestion ( #6405 )
...
* Fix #6387 : Clean Ingestion
* postgres fix
* Fixed Location in Sample Data
2022-07-29 08:51:58 +02:00
Francisco J. Jurado Moreno
ae491f747f
[TASK-6295] Reduce memory footprint for S3 ingestion ( #6308 )
...
* Reduce memory footprint for parquet & CSV formats
* Add pagination, remove local var
* Add jsonl parser
2022-07-25 07:24:57 +02:00
Onkar Ravgan
8c9dc91ccf
Refactored Datalake and Deltalake for Topology ( #6034 )
...
* rebasing with main
* refactored deltalake for topology
* using requests instead of urllib
* formatting fixes
Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-07-19 06:37:27 +02:00
Mayur Singal
4b5b184177
Fix #6091 : Fix Datalake arrays must be of the same length ( #6092 )
2022-07-17 18:26:54 +02:00
Abhishek Pandey
e8975aac01
datalake-csv-files-ingestion-added ( #5343 )
...
datalake-csv-files-ingestion-added (#5343 )
2022-06-15 08:57:21 +02:00
