Pere Miquel Brull
3b7ae73473
Airflow e2e integration test ( #9363 )
...
* Prep airflow operator integration tests
* Add integration test to Makefile
2022-12-16 19:52:12 -08:00
Pere Miquel Brull
0963eac48e
Fix #9182 - Airflow Lineage Operator & Airflow lineage state ( #9206 )
2022-12-10 19:54:41 +01:00
Onkar Ravgan
35efd49256
Added control for DBT descriptions ( #7653 )
...
* Added control for DBT descriptions
* Fixed tests
* Added UI changes
* fixed maven ci tests
* Java formatting changes
* ui review fixes
* Fixed pytests
* Fixed python integration tests
* fixed airflow tests
Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-09-26 16:19:47 +05:30
Pere Miquel Brull
7560635114
Fix #6280 - Bump DeltaLake version, tests and docs ( #6307 )
...
Fix #6280 - Bump DeltaLake version, tests and docs (#6307 )
2022-07-24 18:49:15 +02:00
Snyk bot
c717cb1545
fix: ingestion/tests/integration/lineage/airflow/requirements.txt to reduce vulnerabilities ( #5178 )
...
The following vulnerabilities are fixed by pinning transitive dependencies:
- https://snyk.io/vuln/SNYK-PYTHON-FLASKAPPBUILDER-2433100
2022-06-08 13:36:31 -07:00
Pere Miquel Brull
9138c70dee
Fix #4696 - Only store URL suffixes when extracting metadata ( #5281 )
...
* Only store URL suffix
* Fix test service name
2022-06-03 11:43:40 +02:00
Pere Miquel Brull
02c2dd9494
Fix #4689 - Rename FQDN to FQN and clean unused code ( #5165 )
2022-05-26 21:00:18 +02:00
Pere Miquel Brull
04421901c0
[WIP] Fix #4610 - Pipeline Tasks removal ( #4615 )
...
Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
2022-05-07 21:10:35 -07:00
Pere Miquel Brull
2aecebd595
Fix #4189 - Fix airflow lineage callback ( #4191 )
...
Fix #4189 - Fix airflow lineage callback (#4191 )
2022-04-18 15:13:26 +02:00
Pere Miquel Brull
6768bf708d
Fix Py tests ( #4122 )
...
Fix Py tests (#4122 )
2022-04-14 11:22:39 +02:00
Sriharsha Chintalapani
be836e5404
Fix #4071 : PUT IngestionPipeline missing property & error message ( #4085 )
...
Fix #4071 : PUT IngestionPipeline missing property & error message (#4085 )
2022-04-13 08:40:21 +02:00
Pere Miquel Brull
93525aea0d
Fix #3823 & Fix generate default Enum field ( #3909 )
...
* Fix sample data DAG
* Fix callback imports
* Use --set-default-enum-member in generate
* Format
* Add faulty merge hard_delete
* Fix airflow lineage, improve naming and fix lineage tests
* Add mysql url test
* Add mysql url test
* Update CI name
* Fix test ometa endpoint
* Format
* Fix metadata config
2022-04-07 18:22:50 +05:30
Pere Miquel Brull
e2539c5e83
Fix #3844 - First iteration for deprecating MetadataServerConfig ( #3853 )
...
* Style
* deprecate MetadataServerConfig
* Remove audience from Okta
2022-04-05 18:02:45 +05:30
Sriharsha Chintalapani
b14c8dc2c4
Issue-3685: Variable based separator used for fullyQualifiedName instead of hardcoded . for Python and make the separator to : ( #3778 )
...
* Issue-3685: Variable based separator used for fullyQualifiedName instead of hardcoded . for Python and make the sepearator to :
* Fix failing test
* Use colon for run_local_docker validation
* Update tests FQDN
* Update tests FQDN
Co-authored-by: Sachin-chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: pmbrull <peremiquelbrull@gmail.com>
2022-03-31 19:20:27 +05:30
Pere Miquel Brull
548a0ab722
Fix #3437 & #3186 - Airflow lineage Task Group & Tests ( #3460 )
2022-03-21 10:29:49 -07:00
Snyk bot
8132b91c30
fix: ingestion/tests/integration/lineage/airflow/requirements.txt to reduce vulnerabilities ( #3130 )
...
The following vulnerabilities are fixed by pinning transitive dependencies:
- https://snyk.io/vuln/SNYK-PYTHON-APACHEAIRFLOW-2413442
- https://snyk.io/vuln/SNYK-PYTHON-APACHEAIRFLOW-2414089
- https://snyk.io/vuln/SNYK-PYTHON-WTFORMS-40581
2022-03-04 07:49:58 +01:00
Alberto Miorin
fe5618c8f1
Fix #3037 : metadata --version doesn't work ( #3038 )
2022-03-01 12:19:36 +01:00
Pere Miquel Brull
76f4ccd590
Fix #2735 - Simplify Airflow properties extraction ( #2749 )
...
* Simplify lineage properties extraction
* Add network name
* Format
2022-02-14 16:53:42 +01:00
Pere Miquel Brull
1b2ea54d4f
Fix #2316 - Append Tasks & Add Status in Airflow ( #2738 )
...
* Add license
* Add date to timestamp helper
* Prepare pipeline status operation
* Update and clean tasks from client
* Update tasks from client
* Check if pipeline is empty
* Keep all current pipeline info
* Append and clean tasks
* format
* Add status information
* Test pipelineStatus update
* Update task on clear
* Log status on callback
* Update lineage and status docs
* Update lineage docs
* Format
* Logic to handle DAG status
* Lint and format
* Update lineage tests
2022-02-13 17:51:25 +01:00
Vivek Ratnavel Subramanian
e6343a79d7
Fix #2555 : PUT APIs should only allow admin and owners to update a resource in secure env ( #2608 )
2022-02-04 12:39:08 -08:00
Robert Pack
67a1c23482
Fix #2348 : Harmonize request titles in json schemas
2022-01-31 16:29:56 -08:00
Pere Miquel Brull
05c8d1ebb6
Fix #2153 - Lineage date parsing ( #2387 )
...
* Check for start_date
* Fix date keys and remove repr
* Add date management tests
2022-01-24 12:29:16 -08:00
Sriharsha Chintalapani
10e71a7e09
Fix #2340 : Ingestion pipeline deployment refactor ( #2341 )
...
* Fix #2340 : Ingestion pipeline deployment refactor
2022-01-21 22:06:14 -08:00
Pere Miquel Brull
2728041d0f
[issue-2213] - Airflow lineage refactor ( #2218 )
...
* Fix ometa logging
* Refactor airflow lineage
* Refactor airflow lineage
* Add airflow lineage integration tests
* Fix circular dependency
2022-01-14 13:00:33 -08:00
