48 Commits

Author SHA1 Message Date
Nahuel
8f1ef49cdb
Fix issue ingesting Kafka when the basic auth is enabled in the Schema Registry (#7203) 2022-09-04 15:59:11 +02:00
Pere Miquel Brull
613a329f40
Fix #7089 - Add Redpanda connector (#7092)
Fix #7089 - Add Redpanda connector (#7092)
2022-09-01 06:39:27 +02:00
Pere Miquel Brull
d791fe8289
Fix #6816 - Add supportsQueryComment & Fix #7078 - Athena Session Token (#7079)
* Athena with aws session token

* add supports query comment property to the required db services

* Hide query comment toggle
2022-08-31 14:03:54 +02:00
Nahuel
4c6f0c7bda
Fix#6027: Fix logging in metadata package modules except ingestion and orm_profiler (#6970)
* Fix logging in metadata package modules except ingestion and orm_profiler

* Reduce log level to _get_columns_with_constraints method
2022-08-29 06:46:06 +02:00
Pere Miquel Brull
6cebf37b6c
Fix #6906 - Add Nifi connector (#6956)
* Add Nifi connector

* Mock token

* Format

* pass verify param
2022-08-26 21:50:01 +02:00
Abhishek Pandey
a6a7662517
dagster-connector-added (#6513)
* dagster-connector-added

* code-formatted

* dagster-comments-removed

* dagster-comments-removed

* dagster-unittest-added

* code-formatted-pr-comments-resolved

* improved-code-quality

* increased-py-test-time-out

* improved-dagster-tests

* improved-dagster-tests

* reverted-pytests-timeout

* dagster-pytest-improved

* added-schema-in-omdcore

* updated-schema-in-omd-core

* ui-schema-updated-omd-core

* updated-dagster-schema

* dagter-schema-test-updated
2022-08-13 15:18:14 +05:30
Onkar Ravgan
af33cc0a31
Fixed Tableau ingestion issue (#6671)
* Fixed Tableau ingestion issue

* revmoved doc string

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-08-09 23:55:32 +05:30
Pere Miquel Brull
e2916b2521
Organise ingestion module (#6515)
Organise ingestion module (#6515)
2022-08-03 09:55:50 +02:00
Teddy
a7fffc778c
Fixes #6476 -- Multiple db connections opened (#6478)
* Fixed scoped_session factory creation

* Fixed py format

* Added comment for GC + added explicit QueuePool definition
2022-08-01 13:42:35 +02:00
Teddy
6397b6a0b1
Fixes #6325 -- Implement multithreading for metrics computation (#6406)
* Added tests for multithreading SQA interface

* Added multithread support for metric computation

* Added thread ID to log debuger

* Cleaned up tests

* Fixed python formatting issues

* Added non blocking result processing + threadCount in config file to set numbers of threads

* Added frontend input field to set number of threads

* Fixed code smell, bug and comments from reviewer
2022-07-29 10:41:53 +02:00
Mayur Singal
c6efe8c142
Fix #6158: Fivetran Connector Added (#6329)
* Fivetran Connector Added

* Fivetran unit test added

* Fixed Unit Tests

* Added Fivetran Docs & Added Fivetran in Pipeline Listing
2022-07-27 20:50:56 +05:30
Pere Miquel Brull
7560635114
Fix #6280 - Bump DeltaLake version, tests and docs (#6307)
Fix #6280 - Bump DeltaLake version, tests and docs (#6307)
2022-07-24 18:49:15 +02:00
Pere Miquel Brull
810e6c7dfc
Fix #5822 - Remove looker env and rename props (#5700)
Fix #5822 - Remove looker env and rename props (#5700)
2022-07-24 16:02:26 +02:00
Francisco J. Jurado Moreno
78dd44ed32
[TASK-6140] Add OMeta header to SQLAlchemy-based queries (#6293)
[TASK-6140] Add OMeta header to SQLAlchemy-based queries (#6293)
2022-07-23 12:31:26 +02:00
Onkar Ravgan
8c9dc91ccf
Refactored Datalake and Deltalake for Topology (#6034)
* rebasing with main

* refactored deltalake for topology

* using requests instead of urllib

* formatting fixes

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-07-19 06:37:27 +02:00
Pere Miquel Brull
89a026b022
Fix #5688 - MlFlow deployment fails from UI (#5691)
* Fix mlflow from UI

* format
2022-06-28 14:58:38 +02:00
Milan Bariya
338d1efe40
FIX 5419: Glue pipeline connector not working (#5467)
FIX 5419: Glue pipeline connector not working (#5467)
2022-06-27 11:38:41 +02:00
Sriharsha Chintalapani
710e782f33
Fix #5455: Ingestion from a kafka topic fails when sample data is enabled (#5536)
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2022-06-24 20:39:41 -07:00
Pere Miquel Brull
bb4ac68ee3
Add backend connection for airflow dag extraction 2022-06-24 20:01:50 +02:00
Pere Miquel Brull
0144a1d46a
Fix #5423 & #3412 - Looker Lineage and Usage (#5581)
Fix #5423 & #3412 - Looker Lineage and Usage (#5581)
2022-06-23 13:01:40 +02:00
Onkar Ravgan
21dac15176
Added Mode Connector (#5475)
* rebase with main

* Added lineage for mode

* added lineagerunner and enum constants

* added host port

* refactored mode according to new dashboard topology

* checkstyle fixes

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-06-19 23:59:38 +05:30
Abhishek Pandey
196b023643
airflow-test-connection-fixed (#5502)
* airflow-test-connection-fixed

* airbyte-test-connection-added
2022-06-17 23:08:18 +05:30
Abhishek Pandey
e8975aac01
datalake-csv-files-ingestion-added (#5343)
datalake-csv-files-ingestion-added (#5343)
2022-06-15 08:57:21 +02:00
Pere Miquel Brull
d9d30b2cba
Fix #5416 - Revisit Looker Test Connection (#5422)
Fix #5416 - Revisit Looker Test Connection (#5422)
2022-06-13 08:07:04 +02:00
Onkar Ravgan
a5ff3a3499
Removed connectionOptions dictionary (#5264)
Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-06-02 15:07:22 +02:00
Onkar Ravgan
4671ca00b0
Refactored PowerBi Connector (#5107)
Refactored PowerBi Connector (#5107)
2022-06-01 10:36:12 +02:00
Pere Miquel Brull
0f5777a9fa
Fix #4483 Airflow REST stage file loc; Fix #4738 disable rest_on_return (#5138)
Fix #4483 Airflow REST stage file loc; Fix #4738 disable rest_on_return (#5138)
2022-05-26 09:21:36 +02:00
Abhishek Pandey
591a65d161
updated-json-schema-types (#5047)
* updated-json-schema-types

* code-formatted

* gsc-credential-format-updated
2022-05-25 11:53:55 +02:00
Pere Miquel Brull
8e121fb306
Fix #4096 - PipelineService JSON refactor & Airflow metadata source connector (#5125)
Fix #4096 - PipelineService JSON refactor & Airflow metadata source connector (#5125)
2022-05-25 08:35:16 +02:00
Pere Miquel Brull
c6915ca510
Fix #4923 - Presto ingestion & SQLAlchemy test connection refactor (#4946)
Fix #4923 - Presto ingestion & SQLAlchemy test connection refactor (#4946)
2022-05-14 06:44:43 +02:00
Mayur Singal
862b08ef2c
Fix #4844: Improved Databricks get connection (#4906) 2022-05-12 11:58:34 +02:00
Pere Miquel Brull
6fdd1c4691
Fix #4780 & #4739 - ConnectionArguments with inner dict (#4798) 2022-05-09 11:28:47 -07:00
Pere Miquel Brull
79dbd18508
Import kafka modules inside function (#4805)
* Import kafka inside function

* Format
2022-05-09 15:36:24 +02:00
Sriharsha Chintalapani
34858f3b32
Topics: Add Sample data (#4787)
* Topics: Add Sample data

* Topics: Add Sample data

* Topics: Add Sample data

* Topics: Add Sample data

* Topics: Add Sample data
2022-05-09 12:50:54 +02:00
Pere Miquel Brull
a22ff4627f
Fix #4728 - Fix Kafka test connection (#4730)
* Fix Kafka connection

* Update defaults and required values

* Update test connection

* Add filter tests

* Fix kafka tests
2022-05-06 08:59:05 +02:00
codingwithabhi
4aa5fbce04
tableau-test-connection-added (#4673)
* tableau-test-connection-added

* tableau-test-connection-fixed

* test-connection-optimised

* test-connection-optimised
2022-05-05 06:47:08 +02:00
codingwithabhi
e4d3f0b376
Issue 4406 superset (#4708)
* superset-test-connection-fixed

* optimized-test-connection

* code-formatted
2022-05-04 17:23:42 +02:00
Mayur Singal
450fb2b132
SampleData Usage Fix (#4398)
* SampleData Test Connection & Usage Fix

* Fixed Pytest
2022-05-04 16:45:49 +02:00
codingwithabhi
6bd587be8d
redash-test-connection-completed (#4672)
* redash-test-connection-completed

* code-smell-removed

* docstring-added-for-non-sqlalchemy-service-client
2022-05-04 16:25:46 +02:00
codingwithabhi
6cff9620b2
metabase-test-connection-completed (#4671)
* metabase-test-connection-completed

* debug-traceback-added
2022-05-04 12:56:33 +02:00
codingwithabhi
897e4ff765
snowflake-private-key-fix-via-ui (#4492)
* snowflake-private-key-fix-via-ui

* removed-overriding-test-connection

* code-formatted
2022-04-26 17:10:10 +05:30
Ayush Shah
7c8c51e862
Added consumer config in kafka (#4365) 2022-04-23 08:24:25 -07:00
codingwithabhi
32d392e0cb
deltalake-test-connection-fixed (#4387) 2022-04-23 00:40:44 -07:00
Mayur Singal
1c4bea4613
Salesforce Test Connection Fix (#4376) 2022-04-22 09:34:39 -07:00
Mayur Singal
bbf79d3222
Refractor test connection for glue & dynamodb (#4373)
* Refractor test connection for glue & dynamodb

* Renamed test_connection_client
2022-04-22 17:23:42 +02:00
Ayush Shah
32a16b059d
Modify Snowflake and Databricks schema (#4367) 2022-04-22 20:07:06 +05:30
Pere Miquel Brull
ca0b26dded
Fix #4313 - Add title to connection schemas; Fix #4335 - AWS Credentials (#4338)
Fix #4313 - Add title to connection schemas; Fix #4335 - AWS Credentials (#4338)
2022-04-22 11:25:05 +02:00
Mayur Singal
db0e34c709
Fixing Test Connection for Dynamo & Glue (#4316)
* Fixing Test Connection for Dynamo

* Fixed Glue Connector

* renamed engine to connection

* Fixed the return signature

* Added dataclass
2022-04-22 11:30:59 +05:30