2916 Commits

Author SHA1 Message Date
Pere Miquel Brull
26efe40e87
Add metadata main (#8247) 2022-10-19 10:02:36 +02:00
Milan Bariya
1565aa7733
FIX: Trino&Presto catalogs as databases (#8189)
* FIX: Trino&Presto catalogs as databases

* FIX: Trino&Presto catalogs as databases

* Change based on comments
2022-10-18 20:00:17 +05:30
NiharDoshi99
84a89d7181
Added DOMO Pipeline (#8133) 2022-10-18 17:52:28 +05:30
Milan Bariya
5b3c877dd1
FIX:connection failed because of having '@' character in uname and pass (#8172) 2022-10-17 16:57:55 +05:30
Pere Miquel Brull
d48fd468d4
Tag PATCH + Cleanup of helpers methods (#8150)
* cleanup

* lint

* Add tag patch

* Fix rename

* Dont kill tests
2022-10-15 14:56:30 +02:00
Nahuel
b69581299f
Fix SonarCloud coverage for Python code (#8144) 2022-10-14 15:17:40 +02:00
Mayur Singal
b55a46f4de
Fix #5492: Fix Trino Comments (#8113) 2022-10-14 06:39:31 +02:00
Milan Bariya
e7a3770240
Add: databricks usage and lineage added (#7386) 2022-10-14 07:06:28 +05:30
NiharDoshi99
7975f0e6fe
Pylint Fix (#8134)
* Pylint Fix

* changes as per comment
2022-10-13 14:48:28 +02:00
Nahuel
48905a7f6e
Fix#7532: Allow overriding service connection config when ingesting from CLI (#8091)
* Allow overriding service connection config when ingesting from CLI

* Rename field in JSON schema

* Address comment
2022-10-13 12:34:03 +02:00
Pere Miquel Brull
615b1a9647
Add Airflow deps installation (#8131) 2022-10-13 11:06:27 +02:00
NiharDoshi99
0e44f7d35d
Fixed: Intergrated DOMO Connector APIs (#7989) 2022-10-13 10:55:48 +05:30
Ayush Shah
aa9ef323e2
FIx Nonetype Bigquery Taxonomy (#8110) 2022-10-12 19:54:13 +05:30
Ayush Shah
58765379a1
Fix Policy Tags from diff project ids (#8004) 2022-10-12 15:17:21 +05:30
Onkar Ravgan
92ddda2c15
Fixed DBT snowflake case sensitivty and dbt owner from manifest (#8052)
* Fixed snowflake case and dbt owner manifest

* Merged main after linting

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-10-12 14:59:17 +05:30
Milan Bariya
7aaaddf7cc
FIX: Deleted label not appearing for bigquery (#8090) 2022-10-12 12:55:12 +05:30
Pere Miquel Brull
4b3ca065c4
Fix #3394 - Pylint fixes and add it to CI (#8084)
Fix #3394 - Pylint fixes and add it to CI (#8084)
2022-10-11 17:15:02 +02:00
Teddy
f883863b8a
Fixes #7490 - Split Profiler and TestSuite Interface (#8032)
* Clean up test suite workflow and interface

* Fixed tests

* Split profiler and testSuite interfaces

* Cleaned up workflows and runners

* Fixed code formatting

* - remove old code
- remove `table` attribute used for testing and used mock instead

* Fixed execution bugs from refactor

* Fixed static type checking for profiler/api/workflow.py

* Fixed linting

* Added __init__ files
2022-10-11 15:57:25 +02:00
Ashish Gupta
8ddab79dc1
UI :- Remove rc-tree and react-table dependencies (#8049)
* Remove rc-tree and react-table dependencies

* fix unit test issue

* File Formatted

Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-10-11 15:42:58 +05:30
Teddy
3b7f576d04
Fixed linting for Profiler (#7922)
* - Fixed linting
- Added logic to skip partition check when engine is not BQ
- Added ingestion partition logic to testSuite

* Fixed python formating

* Fixed test for BQ partition
2022-10-11 09:36:36 +02:00
Mayur Singal
22ac150d7b
Database Source Lint (#8034) 2022-10-11 13:05:08 +05:30
Nahuel
806dcea0de
Fix Superset REST client requests (#8058) 2022-10-11 08:23:26 +02:00
Pere Miquel Brull
339abc5bf3
Prepare Ingestion Base Docker image (#8065)
Prepare Ingestion Base Docker image (#8065)
2022-10-11 07:50:49 +02:00
Teddy
3c093f65fc
Fixes test suite pylint (#7937)
* Fixed linting for testSuite

* fixed python format

* Fixed conflict
2022-10-11 06:55:14 +02:00
Teddy
d0984c5a43
Fixes lineage linting (#7956)
* Fixed pylint

* Fixed python formating
2022-10-11 06:55:08 +02:00
Mayur Singal
a2894f7990
Dashboard Source Lint (#7987) 2022-10-11 06:50:55 +02:00
Pere Miquel Brull
f031293af0
Fix Pylint ingestion/bulksink and ingestion/stage (#8030)
* Pylint ingestion bulksink

* Pylint stage

* Remove test

* Remove stage
2022-10-10 14:06:43 +02:00
Onkar Ravgan
107eeef8c7
Added fixes according to pylint (#8009)
Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-10-10 12:53:47 +02:00
Nahuel
5c499d2a7a
Add CLI E2E tests for MySQL (#8041)
* Add CLI E2E tests for MySQL

* Fix setup.py and pylint

* Add missing doc and update code after pylint refactor
2022-10-10 11:36:20 +02:00
Pere Miquel Brull
b6ff10f781
Pylint ingestion models (#8029) 2022-10-10 06:51:24 +02:00
Pere Miquel Brull
ab2fabee63
Pylint api (#8040) 2022-10-10 06:50:28 +02:00
Mayur Singal
13f6f79f30
Metadata Source Lint (#8018) 2022-10-10 06:46:59 +02:00
Mayur Singal
b8e989af6c
Pipeline Source Lint (#8002) 2022-10-10 06:46:31 +02:00
Mayur Singal
4e51c70dcc
Messaging & MlModel Source Lint (#8008)
* Messaging Source Lint

* MlModel Source Lint
2022-10-10 06:46:24 +02:00
Teddy
90c569eadc
Removed click.Tuple() and used nargs instead (#8035) 2022-10-07 18:43:07 +02:00
Sriharsha Chintalapani
5be142959a
Fix #8036: Superset ingestion fails during the lineage fetch for charts (#8037) 2022-10-07 18:42:46 +02:00
Pere Miquel Brull
783557e6da
Fix Pylint for ingestion/sink (#7961)
* pylint on sink

* Add missing registration
2022-10-07 07:02:27 +02:00
Teddy
cd6b3d6b8a
Removes support database for Trino Connection (#7973)
* Removes support database for Trino Connection

* Added logic to set catalog or database attribute for connection if not already set
2022-10-06 10:39:23 +02:00
Teddy
e244a241c3
added logic to clean up quotes from table constraints (#7959) 2022-10-05 16:09:33 +02:00
Milan Bariya
cd43f1fdcf
Fix: connector for hive has double dash (#7953) 2022-10-05 12:01:13 +02:00
Teddy
15f7c4aa41
Fix param name for median test (#7942)
* Fixed param name for median test

* Fixed unite test for median DQ
2022-10-05 06:32:28 +02:00
Teddy
64fdebe491
Fix #7911 -- Trino profiler query (#7920) 2022-10-04 21:22:13 +02:00
Pere Miquel Brull
9c64aedd77
Fix pylint in ometa_api (#7899) 2022-10-04 16:57:07 +02:00
Ayush Shah
ad53c509ba
Fix Bigquery ADC diff projectid (#7935) 2022-10-04 19:58:17 +05:30
Sriharsha Chintalapani
8efc0370e5
Fix #7904: Filter out old tasks which doesn't have dag runs while publishing status (#7905)
* Fix #7904: Filter out old tasks which doesn't have dag runs while publishing status

* Fix #7904: Filter out old tasks which doesn't have dag runs while publishing status
2022-10-04 06:34:55 +02:00
Abhishek Pandey
566602373f
datalake-gcs-performance-improved (#7609) 2022-10-04 09:52:10 +05:30
Mayur Singal
7ecc09c17c
Glue Pipeline Name Fix (#7892) 2022-10-03 19:38:10 +05:30
Teddy
63b5233fd3
Added util command to overwrite dag_generated_config path in DAG file (#7831)
* Added util command to overwrite dag_generated_config path in DAG file

* Added logic to only write file on change
2022-10-03 09:24:47 +02:00
Teddy
819188cfc1
Fixed mysql BLOB type when ingesting sample data (#7813) 2022-10-02 16:44:41 -07:00
Mayur Singal
ca51fa0245
Fix #7801: Fixed Glue Pipeline (#7832)
Co-authored-by: Apple <apple@Apples-MacBook-Pro.local>
2022-10-02 16:43:35 -07:00