984 Commits

Author SHA1 Message Date
Pere Miquel Brull
dbdd3caf1d
Fix #4332 - Athena LEN & hist concat cast + disable (#4371) 2022-04-22 20:29:59 +05:30
Ayush Shah
a94bb5fc8e
Fixes Tableau + fix Charts not getting ingested as part of dashboard (#4327)
* Fixes Tableau + fix Charts not getting ingested as part of dashboard

* pytest fix

* Fix Tableau json config
2022-04-22 12:49:55 +02:00
Pere Miquel Brull
ca0b26dded
Fix #4313 - Add title to connection schemas; Fix #4335 - AWS Credentials (#4338)
Fix #4313 - Add title to connection schemas; Fix #4335 - AWS Credentials (#4338)
2022-04-22 11:25:05 +02:00
Mayur Singal
db0e34c709
Fixing Test Connection for Dynamo & Glue (#4316)
* Fixing Test Connection for Dynamo

* Fixed Glue Connector

* renamed engine to connection

* Fixed the return signature

* Added dataclass
2022-04-22 11:30:59 +05:30
Onkar Ravgan
0536cdedcf
Fixes for vertica password issue and added connection url test (#4323)
* rebase with main

* optimized password logic and added tests

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-04-22 07:38:35 +02:00
Pere Miquel Brull
2444d3de3d
Fix #4235 - Run data profiler workflows from Airflow REST (#4325)
Fix #4235 - Run data profiler workflows from Airflow REST (#4325)
2022-04-21 17:53:29 +02:00
Ayush Shah
09f01fc7e3
Issue 4237 - fixes empty dictionary (#4239) 2022-04-19 21:22:46 +05:30
Pere Miquel Brull
256b16d877
Fix #4032 - Bigquery properties & GCS Credentials (#4202)
Fix #4032 - Bigquery properties & GCS Credentials (#4202)
2022-04-19 12:31:34 +02:00
Milan Bariya
924e211ba1
ISSUE-4076: Use the central get_engine method for usage (#4180)
* ISSUE-4076: Use the central get_engine method for usage

* ISSUE-4076: Use the central get_engine method for usage

* ISSUE-4076: Use the central get_engine method for usage

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2022-04-19 12:35:01 +05:30
Pere Miquel Brull
63415952e3
Fix sample data (#4200)
Fix sample data (#4200)
2022-04-18 19:00:36 +02:00
pmbrull
93f36ec655 Merge remote-tracking branch 'upstream/main' into issue-4131-v2 2022-04-18 17:39:07 +02:00
pmbrull
0c55a7ccc1 Add database required fields 2022-04-18 17:38:34 +02:00
pmbrull
c7fe4f8968 Add dashboard required fields 2022-04-18 16:45:33 +02:00
pmbrull
188e5e886e Prepare source parsing tests 2022-04-18 16:41:29 +02:00
Pere Miquel Brull
2aecebd595
Fix #4189 - Fix airflow lineage callback (#4191)
Fix #4189 - Fix airflow lineage callback (#4191)
2022-04-18 15:13:26 +02:00
pmbrull
734349a137 Merge remote-tracking branch 'upstream/main' into issue-4131-v2 2022-04-18 14:34:46 +02:00
Pere Miquel Brull
3f4129d239
Prepare custom encoder for SecretStr (#4186) 2022-04-18 14:43:22 +05:30
codingwithabhi
8115c2ece4
issue-4135: updated-mysql-connection-json (#4136) 2022-04-14 14:16:09 -07:00
Mayur Singal
c48bc18065
Fix Hive & DB2 (#4147) 2022-04-14 10:27:05 -07:00
pmbrull
9faa93213f Prepare tests 2022-04-14 17:33:10 +02:00
Pere Miquel Brull
6768bf708d
Fix Py tests (#4122)
Fix Py tests (#4122)
2022-04-14 11:22:39 +02:00
Pere Miquel Brull
43be01b92e
Fix #4091 & #3948 - Create services correctly (#4101) 2022-04-13 18:04:52 +02:00
Ayush Shah
1f3667b46f
Fix Python test (#4095) 2022-04-13 21:03:33 +05:30
Sriharsha Chintalapani
be836e5404
Fix #4071: PUT IngestionPipeline missing property & error message (#4085)
Fix #4071: PUT IngestionPipeline missing property & error message (#4085)
2022-04-13 08:40:21 +02:00
Milan Bariya
5e6e878d8e
ISSUE-3224: Add unit test for mysql (#4075)
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-04-12 13:10:18 -07:00
Ayush Shah
32004021c8
Schema Default value changed for sample data (#4067) 2022-04-12 19:28:32 +05:30
Pere Miquel Brull
6a6507e754
Fix #3962 - Profiler uses DatabaseSchema & Sample Data fix (#4056) 2022-04-12 13:40:59 +05:30
Sriharsha Chintalapani
96dd94eab7
Merge pull request #4022 from MilanBariya/issue-3266
ISSUE-3266: Add unit test for vertica
2022-04-11 13:01:05 -07:00
Sriharsha Chintalapani
9479b49af6
Merge pull request #4030 from ulixius9/issue-3830
Fix #3830: Fixed Query parser
2022-04-11 08:12:13 -07:00
Milan Bariya
b64384e84a Add unit test for dynamodb 2022-04-11 19:33:27 +05:30
ulixius9
38ee0eca49 Fix #3830: Fixed Query parser 2022-04-11 18:59:36 +05:30
Milan Bariya
4fdb99bb26 ISSUE-3266: Add unit test for vertica 2022-04-11 17:57:05 +05:30
Milan Bariya
d73c7d9b44 ISSUE-3266: Add unit test for vertica 2022-04-11 15:17:22 +05:30
Mayur Singal
a3ceea6b63
Hive Source Fix (#3919)
* Hive Source Fix

* Added auth option test & unified unit test
2022-04-08 22:29:48 +05:30
Pere Miquel Brull
06a3e4c989
Fix #3825 - Schema Name, SQL Source FQDN & ORM Profiler (#3942)
* Fix db schema name

* Fix sqlite connection

* Correctly register scanned tables

* improve sqlite connection

* Adapt schemas on ORM profiler

* Format
2022-04-08 19:28:10 +05:30
Milan Bariya
4d15d0f91e
ISSUE-3811: Add unit test for clickhouse (#3949) 2022-04-08 19:26:02 +05:30
dvecerdea
230fc256d6
Refactor Trino Connector and allow proxy setup in trino connection config (#3810)
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2022-04-07 12:50:37 -07:00
Ayush Shah
7c3a6082a6
Refactor - Redshift Connection (#3920) 2022-04-07 08:52:38 -07:00
Mayur Singal
3651efd7f5
Databricks Source Fix (#3917)
Databricks Source Fix (#3917)
2022-04-07 16:40:50 +02:00
Pere Miquel Brull
93525aea0d
Fix #3823 & Fix generate default Enum field (#3909)
* Fix sample data DAG

* Fix callback imports

* Use --set-default-enum-member in generate

* Format

* Add faulty merge hard_delete

* Fix airflow lineage, improve naming and fix lineage tests

* Add mysql url test

* Add mysql url test

* Update CI name

* Fix test ometa endpoint

* Format

* Fix metadata config
2022-04-07 18:22:50 +05:30
Pere Miquel Brull
bd4071bd64
Fix #3826 & #3886 - Profiler workflow & filter pattern (#3893)
Fix #3826 & #3886 - Profiler workflow & filter pattern (#3893)
2022-04-06 17:05:00 +02:00
Pere Miquel Brull
63533eb388
Fix for connectors based on refactoring of schemas V2 (#3870)
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-04-05 18:33:25 -07:00
Pere Miquel Brull
b3480693e4
Fix #3824 - OMeta Schema and JSON Connections (#3861)
Fix #3824 - OMeta Schema and JSON Connections (#3861)
2022-04-05 21:20:39 +02:00
Pere Miquel Brull
e2539c5e83
Fix #3844 - First iteration for deprecating MetadataServerConfig (#3853)
* Style

* deprecate MetadataServerConfig

* Remove audience from Okta
2022-04-05 18:02:45 +05:30
Onkar Ravgan
01b246f586
ISSUE-3485: Added DynamoDB Unit Test (#3488)
* ISSUE-3485: Added DynamoDB Unit Test

* Added more tests and optimized code

* Re-added list check condition

* Added file sink test

* Fixed file path

* added assert for table and column names

* Modified exception block and optimized test cases

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-03-31 20:38:17 +05:30
Sriharsha Chintalapani
b14c8dc2c4
Issue-3685: Variable based separator used for fullyQualifiedName instead of hardcoded . for Python and make the separator to : (#3778)
* Issue-3685: Variable based separator used for fullyQualifiedName instead of hardcoded . for Python and make the sepearator to :

* Fix failing test

* Use colon for run_local_docker validation

* Update tests FQDN

* Update tests FQDN

Co-authored-by: Sachin-chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: pmbrull <peremiquelbrull@gmail.com>
2022-03-31 19:20:27 +05:30
codingwithabhi
2006b2b8ac
bigquery-unittest-added (#3479)
* bigquery-unittest-added

* mock-added-in-create_bigquery_client

* complex-struct-added

* makefile-updated-to-install-all-packages-for-unittest

* bigquery-unittest-updated-via-file-sink
2022-03-31 14:27:51 +05:30
Pere Miquel Brull
9ced748c4f
Use root for FQDN (#3780) 2022-03-31 12:05:11 +05:30
Sriharsha Chintalapani
f79b251509
Fix #3771: Main branch schemas and ingestion is failing; Fix #2992: Remove ChangeEvents log from ElasticSearch Indexes (#3772) 2022-03-30 18:26:57 -07:00
Pere Miquel Brull
c22381fdc8
Fix #3747 - FQDN model in ometa api (#3748)
Fix #3747 - FQDN model in ometa api (#3748)
2022-03-30 08:54:47 +02:00