186 Commits

Author SHA1 Message Date
Pere Miquel Brull
3cbd0bc1d8
Fix #5379 - Fix service type handling on REST APIs (#5380) 2022-06-08 11:36:10 -07:00
Pere Miquel Brull
8e9d0a73f6
Fix #3573 - Sample Data refactor & ORM converter improvements (#5265)
Fix #3573 - Sample Data refactor & ORM converter improvements (#5265)
2022-06-08 16:10:40 +02:00
Pere Miquel Brull
a57806553a
Fix #4889 - IngestionPipeline reads source connection dynamically; Fix #4888 - Remove force deploy (#5228) 2022-06-02 20:40:53 +02:00
Nahuel
628296d294
Fix #2725: Add IngestionPipeline Task logs (#5152) 2022-05-26 10:08:56 -07:00
Pere Miquel Brull
0f5777a9fa
Fix #4483 Airflow REST stage file loc; Fix #4738 disable rest_on_return (#5138)
Fix #4483 Airflow REST stage file loc; Fix #4738 disable rest_on_return (#5138)
2022-05-26 09:21:36 +02:00
Ayush Shah
93c32ad801
Removed Ingestion Core dependency (#5006) 2022-05-18 00:01:20 +05:30
Pere Miquel Brull
d8383fd0e5
Clean DagRun info (#4877) 2022-05-11 16:02:06 +02:00
Pere Miquel Brull
21ab0ef6f5
Fix #4886 - Overwrite dag generated configs JSON (#4887)
* Update api docs

* Remove deployment check
2022-05-11 15:39:30 +02:00
Pere Miquel Brull
23b95a267b
Fix #4828 - Remove lineage backend from airflow image (#4833)
* Remove callbacks from DAG

* Remove lineage backend from image
2022-05-11 07:36:33 +02:00
Pere Miquel Brull
ef9515bdf3
Fix #4793 - pipelineStatus for unscheduled ingestions (#4799)
* Delete configs

* Check only None

* Format
2022-05-09 15:24:58 +02:00
Pere Miquel Brull
3c4b582136
Revert "Configure entrypoint for airflow plugin (#4733)" (#4761)
This reverts commit 2b1fca377d4c32010b1f2f67bc63bac2ba7ec4b0.
2022-05-06 10:13:01 +02:00
codingwithabhi
ee1dda5366
postgres-selecting-multiple-databases-per-service (#4736) 2022-05-06 06:31:07 +02:00
Vitor Avancini
2b1fca377d
Configure entrypoint for airflow plugin (#4733)
* Configure entrypoint for airflow plugin

* fix formatting
2022-05-06 06:30:12 +02:00
Sriharsha Chintalapani
c0fa190c82
Update README.md 2022-05-04 19:11:02 -07:00
Pere Miquel Brull
3f8f1d1c05
Fix #4589 - Check Airflow REST status; Fix #4525 - Clean REST deps (#4606)
* Add rest status

* Check REST status

* Clean REST deps
2022-05-02 12:27:20 +02:00
Pere Miquel Brull
4ec9f3e117
Validate dag_id argument (#4593) 2022-04-29 09:20:29 -07:00
Pere Miquel Brull
34b90de388
Remove default logger from ingestionpipeline (#4584)
* Remove default logger from ingestionpipeline

* Format
2022-04-29 09:40:05 +02:00
Akash Jain
76b9ee6ee7
updated deploy docs (#4567) 2022-04-29 11:30:59 +05:30
Pere Miquel Brull
0c51ecde63
Fix #2830 - Centralise loggers and update format (#4570)
Fix #2830 - Centralise loggers and update format (#4570)
2022-04-29 06:54:30 +02:00
Pere Miquel Brull
a7328ccf1b
Fix #4491 - PipelineIngestion to support debug (#4564) 2022-04-28 15:50:03 +05:30
Akash Jain
55e1be9ca4
feat: Prepare 0.11.0-snapshot version changes (#4505)
* updated version to 0.11.0-SNAPSHOT

* updated version to 0.11.0.dev* for ingestion

* docker version updates for metadata docker cli

* OM Server Env Variables from #4197 #4261 #4473
2022-04-27 00:41:17 +05:30
Pere Miquel Brull
87a85b085e
Fix usage source type (#4471) 2022-04-25 11:55:12 +02:00
Pere Miquel Brull
dbdd3caf1d
Fix #4332 - Athena LEN & hist concat cast + disable (#4371) 2022-04-22 20:29:59 +05:30
Mayur Singal
db0e34c709
Fixing Test Connection for Dynamo & Glue (#4316)
* Fixing Test Connection for Dynamo

* Fixed Glue Connector

* renamed engine to connection

* Fixed the return signature

* Added dataclass
2022-04-22 11:30:59 +05:30
Pere Miquel Brull
2444d3de3d
Fix #4235 - Run data profiler workflows from Airflow REST (#4325)
Fix #4235 - Run data profiler workflows from Airflow REST (#4325)
2022-04-21 17:53:29 +02:00
Pere Miquel Brull
c3993119f6
Fix #4258 - Check dag bag and discard run id search (#4264)
Fix #4258 - Check dag bag and discard run id search (#4264)
2022-04-20 14:55:33 +02:00
Pere Miquel Brull
e876d01841
prepare usage config (#4279) 2022-04-20 14:10:40 +02:00
Pere Miquel Brull
77af1f2218
Fix sink and secretStr (#4269) 2022-04-20 15:29:56 +05:30
Pere Miquel Brull
098cb865d5
Fix #4257 - Missing airflow params (#4259)
Fix #4257 - Missing airflow params (#4259)
2022-04-20 09:14:27 +02:00
Sriharsha Chintalapani
6b77ab29f2
Fix #4089: Test Connection - Ingestion Resource; Fix #4129: Status -Ingestion Resource (#4215)
* Fix #4089: Test Connection - Ingestion Resource; Fix #4129: Status - Ingestion Resource

* use TestServiceConnectionRequest

* Simplify test service connection

* Add exception info

* Update pipelineStatus

* Deprecate old class

* Use JSON Schema models

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2022-04-19 09:54:10 -07:00
Pere Miquel Brull
256b16d877
Fix #4032 - Bigquery properties & GCS Credentials (#4202)
Fix #4032 - Bigquery properties & GCS Credentials (#4202)
2022-04-19 12:31:34 +02:00
Pere Miquel Brull
3f4129d239
Prepare custom encoder for SecretStr (#4186) 2022-04-18 14:43:22 +05:30
Pere Miquel Brull
c4a6bbdc83
Fix #4146 - Airflow REST delete DAG (#4149) 2022-04-14 12:52:13 -07:00
Pere Miquel Brull
31fbf1a500
Get dag status (#4109) 2022-04-13 18:08:09 +02:00
Pere Miquel Brull
87e5854f25
Fix #4039 - Airflow REST Trigger & Test Connection (#4072)
Fix #4039 - Airflow REST Trigger & Test Connection (#4072)
2022-04-12 17:06:49 +02:00
Pere Miquel Brull
d6b49f070c
Fix #3951 - OpenMetadata Airflow REST (#4034)
Fix #3951 - OpenMetadata Airflow REST (#4034)
2022-04-11 18:38:26 +02:00