157 Commits

Author SHA1 Message Date
codingwithabhi
c6bd06944b
glue-connector-fixed (#3980)
* glue-connector-fixed

* code-formatted

* ran-make-core_bump_version_dev

* config-stored-in-variable
2022-04-09 18:28:53 +05:30
Ayush Shah
fbf5861128
Refactored Superset (#3979) 2022-04-09 18:10:00 +05:30
Ayush Shah
3840eb77c0
Refactoring Connectors - Metabase, MariaDB, Looker (#3977)
* Metabase Fix

* Looker, Metabase and Mariadb

* modified metabase
2022-04-09 16:56:12 +05:30
Onkar Ravgan
d5758b2fb2
Fixed Db2 source config (#3978)
Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-04-09 16:30:18 +05:30
Mayur Singal
53e4403ccd
Fixed MSSQL & Clickhouse Usage (#3976) 2022-04-09 13:50:24 +05:30
codingwithabhi
0e0c3459a8
postgres-connector-fixed (#3970) 2022-04-08 13:28:00 -07:00
Ayush Shah
839d152cc3
PowerBI - Schema Refactoring (#3960)
* Added PowerBI refactoring

* Update powerbi.py

* Update powerbi.py

* Modified filters
2022-04-09 00:38:41 +05:30
Mayur Singal
a3ceea6b63
Hive Source Fix (#3919)
* Hive Source Fix

* Added auth option test & unified unit test
2022-04-08 22:29:48 +05:30
Onkar Ravgan
7b26b5ce24
tableau-connector-fix-added (#3946)
* Fixed Tableau source config

* Review changes

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-04-08 19:23:44 +05:30
codingwithabhi
bb76a2f11b
salesforce-connector-fixed (#3923)
* salesforce-connector-fixed

* salesforce-connection-updated

* ran-make-core_bump_version_dev

* code-formatted

* removed-get_connection_url

* code-formatted

* code-formatted

* code-formatted
2022-04-08 16:08:31 +05:30
Onkar Ravgan
5219bad5e1
singlestore-connector-fix-added (#3937)
* Singlestore config fix
Rebase with main

* Fixed Singlestore source config

* Formating changes

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-04-08 15:09:10 +05:30
Onkar Ravgan
aae29b1675
Fixed vertica source configs (#3910)
Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-04-08 14:13:22 +05:30
codingwithabhi
300e8fec55
snowflake-connector-fix (#3885)
* snowflake-connector-fixed

* schema-filter-fixed

* updated-connection_arguments

* ran-py_format

* removed-FQN-computation

* ran-make-core_bump_version_dev-and-py_format

* ranpy_format
2022-04-08 13:28:54 +05:30
Ayush Shah
7c3a6082a6
Refactor - Redshift Connection (#3920) 2022-04-07 08:52:38 -07:00
Mayur Singal
3651efd7f5
Databricks Source Fix (#3917)
Databricks Source Fix (#3917)
2022-04-07 16:40:50 +02:00
Onkar Ravgan
4dce0a061a
Dynamodb source fix (#3882)
Dynamodb source fix (#3882)
2022-04-07 10:15:47 +02:00
Mayur Singal
d7c73362a7
Clickouse Source Fix (#3891) 2022-04-06 10:20:37 -07:00
Mayur Singal
b709a8055e
Fixed MSSQL Source (#3877)
Fixed MSSQL Source (#3877)
2022-04-06 14:21:46 +02:00
Pere Miquel Brull
63533eb388
Fix for connectors based on refactoring of schemas V2 (#3870)
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-04-05 18:33:25 -07:00
Ayush Shah
07266b58f7
Fix: Added DeltaLake Service type (#3639)
* fix-deltalake

* added deltalake.json

* Bumped up version
2022-03-24 18:42:00 +05:30
Mayur Singal
a38b08882a
Fix #3566: ingesting sample data with space in name (#3586) 2022-03-22 18:29:03 +05:30
Ayush Shah
5cacd33e2e
Standardized Metabase db_service_name across dashboards (#3458)
* Standardized Metabase db_service_name across dashboards

* Resolved Code Smell

* Update metabase.py
2022-03-22 12:21:54 +05:30
Ayush Shah
7c84f063e6
Minor Fixes - Ingestion - Datetime format, Looker fix (#3365)
* Minor Fixes - UTC timezone for usage, tag method modification, looker fix

* Modified core version

* Snowflake Usage Result limit and Makefile ignore env fix
2022-03-11 13:31:42 +05:30
codingwithabhi
37457dc668
table-name-fail-fix- (#3114) 2022-03-03 13:44:34 -08:00
Mayur Singal
3f144a7370
fix #3077: Fixed validation error in ldap (#3079)
* fix #3077: fixed validation error

* removed print

* changes returntype to OMetaUserProfile
2022-03-03 11:11:07 +05:30
codingwithabhi
06d9329ae3
Atlas connector (#2855)
* Fix #1161: Add Atlas Connector

* Atlast Decimal type handled

* Atlas col_data_type handled

* Atlas columns exception handling implemented

* atlas-services-added-into-single-file

* import-updated

* authentication-fix

* atlas-connector-services-added

* code-smell-removed

* code-smell-removed

* code-smell-removed

* file-exist-check-added

* updated-mapping-json-logic

* pr-comment-changes-completed

* atlas-linegae-added

* atlas-linegae-added

* pr-changes-done

* code-formatted

* bug-resolved

* topic-bug-resolved

Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-03-01 11:50:14 +05:30
Ayush Shah
568845b65c
Power BI connector (#3019)
* Power BI Dashboard and Tiles added

* Added Powerbi dependency

* Modified Powerbi debug logs

* Bump up _version.py

* Modified Power BI - added failure status, resolved comments
2022-03-01 10:09:35 +05:30
Mayur Singal
e79be68bea
Fix #2937: added clickhouse usage connector (#2987) 2022-02-26 08:18:49 -08:00
Mayur Singal
4739aaa096
Fix #2905: Added MSSQL Usage Connector (#2948)
* Fix #2509: added mssql usage connector

* removed comment

* removed unused variables

* fixed code smell

* renamed sql variable
2022-02-24 01:01:03 +05:30
Sriharsha Chintalapani
165f5bb0e9
Fix #2819: Kafka connector security settings along with all configs to be retrieved (#2823)
* Fix #2811: Log level option for running metadata

* Fix #2819: Kafka connector security settings along with all configs to be retrieved

* Fix #2819: Kafka connector security settings along with all configs to be retrieved
2022-02-17 15:17:43 +05:30
Mayur Singal
419448a28f
Issue 2727: Added Databricks Connector (#2745)
* Issue #2727: Added Databricks Connector

* optional db and licence info

* make core_bump_version_dev

* host_port fix

* fixed service_type

* change package to sqlalchemy-databricks

* Update databaseService.json

Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-02-17 11:22:13 +05:30
codingwithabhi
088dba3a1f
azuresql-config-added (#2711)
* azuresql-config-added

* rebased-with-main

* tables-ingestion-added

* duplicate-files-removed

* Update azuresql.py

Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-02-17 09:23:07 +05:30
codingwithabhi
9011b27bf6
table-filter-added (#2770)
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-02-16 19:26:40 +05:30
Mayur Singal
5e4cd516bf
issue #2712: Added SingleStore Connector (#2747) 2022-02-15 21:11:46 -08:00
Mayur Singal
88afb17ee8
issue #2611: added clickhouse connector (#2679)
Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
2022-02-10 08:32:01 -08:00
Ayush Shah
4f5f63b226
Issue-2672: Added support for personal access token name and secret and upgraded tableau version (#2682) 2022-02-08 23:02:56 +05:30
Ayush Shah
18d963c3d8
Issue-969: Superset Lineage Implemented (#2659) 2022-02-07 16:16:38 -08:00
codingwithabhi
66c3f43bbb
Fix #2537: Ingestion Connector - DynamoDB 2022-02-07 10:52:11 -08:00
Mayur Singal
2d25da149d
Fix #2626: add db2 connector (#2661)
Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
2022-02-07 10:49:59 -08:00
Ayush Shah
6f5a4eafe7
Bigquery support for env without passing any config (#2160)
* Bigquery support for env without passing any config

* Bigquery File formatted
2022-01-22 22:25:48 -08:00
codingwithabhi
ddf4ae261b
Bigquery credential configuration updated (#2041)
* bigquery-credentials-in-bigquery-json

* ui-changes-reverted

* credentials-moved-to-options

* Update bigquery.json

* Update bigquery.py

* delete-temp-code-added

* delete-temp-code-added

* removed-init-from-config

* Update datatypes_test.py

* Update column_helpers.py

* code-formatted

Co-authored-by: Abhishek <abhishek@Abhisheks-MacBook-Air.local>
Co-authored-by: = <=>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-01-21 17:23:02 +05:30
Matt
2a80caa8fe
Add logic to initialize relationships from seed data during application startup (#2307)
* Add logic to initialize relationships from seed data during application startup

* Remove ingestion related code for access control policies

* Move PolicyEvaluator init to PolicyResource
2022-01-20 22:18:03 -08:00
Sriharsha Chintalapani
0d3ded0742
Data Profiler Integration (#2235)
* Fix 2234: Data profiler integration
2022-01-18 20:25:43 -08:00
Sriharsha Chintalapani
cf6f438531
Fix #1994: Add support for marking dataset entities as deleted (#2141)
* Fix #1994: Add support for marking dataset entities as deleted

* Fix #1994: Add support for marking dataset entities as deleted

* Fix #1994: Add support for marking dataset entities as deleted
2022-01-11 11:42:53 +05:30
Ayush Shah
19e5f737ff
Usage fix - removed service_name from bulk_sink (#1956)
* Query Usage fixed

* Usage updated

* Reformatted change

* Usage files reformatted
2022-01-04 17:12:10 +05:30
Matt
ab4c9ede25
Add ingestion for access_control_policies to support RBAC (#1969) 2021-12-30 08:25:09 -08:00
Matt
f7653f4122
Add support for GCS Lifecycle Policy Ingestion (#1923)
- Add example gcs.json
- Add support for MoveAction and DeleteAction
2021-12-26 20:26:33 -08:00
Matt
4588121520
Add support for DeleteAction in S3 Lifecycle Policy (#1916)
- Support Delete Action for S3
- Add Example s3.json
- Create AWSClient util
- Use AWSClient util in S3 ingestion source
- Remove ambiguity in policy filters by removing array with different types
2021-12-25 16:29:10 -08:00
Sriharsha Chintalapani
485fdea7b1
Fix #1854: Support DBT 1.0 format (#1855)
* Fix #1854: Support DBT 1.0 format
2021-12-20 12:25:26 -08:00
Sriharsha Chintalapani
3c254440a5
Fix #1841: Presto Connector issues (#1842) 2021-12-18 23:27:51 -08:00