2362 Commits

Author SHA1 Message Date
Mayur Singal
2631dbbe7e
Fix #3487: fixed MySQL multiple database ingestion issue (#3489)
* Fix #3487: fixed MySQL multiple database ingestion issue

* moved schma names to prepare

* fixed comment
2022-03-21 17:09:46 +05:30
Mayur Singal
98421f2d56
fix #3423: excluded unwanted query types (#3484) 2022-03-17 09:40:44 -07:00
Ayush Shah
83de95c3f2
Bigquery Custom query for partitioned table and policy tag fix (#3472)
* Bigquery Custom query for partitioned table and policy tag fix

* Added source to tag label

* Modified default partition query as per partition key
2022-03-16 21:05:25 -07:00
Mayur Singal
6254fd8468
Fix #3419: large complex datatype issue fixed (#3449)
* large complex datatype issue fixed

* removed unused variable
2022-03-16 12:21:44 +05:30
Ayush Shah
28a1918ed8
Fixed Looker - Dashboard and Dashboard Elements (#3441)
* Looker fix

* Update looker.py
2022-03-16 11:52:29 +05:30
Pere Miquel Brull
16e82d45de
Fix #3371 - Run Profiler and Tests on a % of the data (#3424)
Fix #3371 - Run Profiler and Tests on a % of the data (#3424)
2022-03-16 06:05:59 +01:00
Mayur Singal
7eaea054c1
Fix #3332: fixed varbyte & time issue (#3401)
* Fix #3332: fixed varbyte & time issue

* added time without timezone

* optimized code

* code_smell_fixed

* removed unused var

* reduced cognitive complexity
2022-03-15 19:48:16 +05:30
codingwithabhi
55c65b5c53
issue-3420: debug-logger-added (#3434) 2022-03-15 19:13:31 +05:30
Mayur Singal
3e92693d34
Fix #3419: Databricks struct, array & map supported (#3426)
* Fix #3419: Databricks struct, array & map supported

* optimized code

* added comments
2022-03-15 19:13:05 +05:30
Onkar Ravgan
ad04136847
Fix #3095: Added support for private key (#3408)
* ISSUE-3095: Add support for private key in snowflake

rebase main branch

* Fix #3095: Added support for private key

* Fix #3095: Added support for private key and review changes

* Fix #3095: Added support for private key and review changes

* changed direct assignment of dictionary

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-03-15 18:19:02 +05:30
Ayush Shah
a6e2ee78c4
Update docker.py (#3430) 2022-03-15 17:18:00 +05:30
codingwithabhi
e043e89dff
ISSUE-3293: optimized-metadata-extraction (#3409)
* ISSUE-3293: optimized-metadata-extraction

* tuple-removed-for-_get_table_comment
2022-03-15 09:22:51 +05:30
Pere Miquel Brull
130bbb0c5c
Fix #3104 - Remove unused imports with pycln (#3370)
Fix #3104 - Remove unused imports with pycln (#3370)
2022-03-14 06:59:15 +01:00
Ayush Shah
7c84f063e6
Minor Fixes - Ingestion - Datetime format, Looker fix (#3365)
* Minor Fixes - UTC timezone for usage, tag method modification, looker fix

* Modified core version

* Snowflake Usage Result limit and Makefile ignore env fix
2022-03-11 13:31:42 +05:30
Pere Miquel Brull
f0dd85b9f7
Fix #3356 - Vertica concat and len (#3358) 2022-03-10 08:01:47 -08:00
Ayush Shah
4f1ef18c2b
Fix Usage from Bulk Sink (#3354)
* Fix Usage from Bulk Sink

* Closing stage in worklow once bulk sink starts
2022-03-10 17:12:42 +05:30
Mayur Singal
40e7d37244
Snowflake optional config (#3319)
* Update AirflowUtils.java

* Update databaseServiceMetadataPipeline.json

* Updated Account and schema description

* snowfliks ui support (#3318)

* added lineage for snowflake

* Formatted file

* initialized metadata_config

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-03-09 17:55:37 +05:30
Ayush Shah
116f8d333d
Airflow Auth fix + Version Downgrade + Increased Docker Min Memory Limit (#3233) 2022-03-08 13:51:07 -08:00
Vivek Ratnavel Subramanian
f0927ecebb
Fix #3236: Add support for secure docker compose with an env file (#3241)
* Fix #3236: Add support for secure docker compose with an env file
2022-03-08 07:05:31 -08:00
Pere Miquel Brull
94d7500216
Fix #3248 & #3251 - Update metrics and column profile (#3262)
Fix #3248 & #3251 - Update metrics and column profile (#3262)
2022-03-08 11:44:39 +01:00
Teddy
6ee31eb21f
Issue-2538: Add Iceberg type for Glue Table + Location Entity (#3210) 2022-03-07 08:27:14 -08:00
Pere Miquel Brull
954908efec
Fix #3217 - Test case props should be unique (#3220)
Fix #3217 - Test case props should be unique (#3220)
2022-03-07 14:43:42 +01:00
Ayush Shah
ce99d52cd7
Fixing Snowflake Query and arrayDataType issue (#3174) 2022-03-07 12:25:01 +05:30
Ayush Shah
23b97f5fe1
Datetime concat issue fixed - Auth Client - Ingestion (#3195)
* Added check for datetime issue concat with float

* Added utc time as google sends expiry in utc datetime
2022-03-07 12:24:48 +05:30
Pere Miquel Brull
2444b884bf
Fix #3150 - Missing count & value length tests (#3193)
Fix #3150 - Missing count & value length tests (#3193)
2022-03-07 07:19:13 +01:00
darth-coder00
4d09c165d7
Feat #2857 - Adding UI support for glossary (#3092) 2022-03-06 21:34:02 -08:00
Pere Miquel Brull
4a752e3ab2
Fix #3151 - Ingestion profiler should use ORM Profiler (#3192) 2022-03-06 15:43:43 -08:00
Mayur Singal
cd6b28df1a
Fix #3066: Superset Auth Fixed (#3156) 2022-03-04 12:02:51 -08:00
codingwithabhi
ad341e8259
fix ISSUE-3137: updated-redash-library-and-function (#3157) 2022-03-04 12:02:08 -08:00
Ayush Shah
0d1fa67116
Added fix for strptime expecting string (#3154) 2022-03-04 23:36:52 +05:30
Pere Miquel Brull
4116233697
Fix #3105 - ColumnValuesToMatchRegex & other fixes (#3149)
Fix #3105 - ColumnValuesToMatchRegex & other fixes (#3149)
2022-03-04 18:11:49 +01:00
Pere Miquel Brull
bd7b91b448
Fix #3112 - col profile safety & sample data (#3142)
Fix #3112 - col profile safety & sample data (#3142)
2022-03-04 13:14:11 +01:00
codingwithabhi
40d3f5e239
column-issue-fixed (#3138)
* column-issue-fixed

* column-type-lookup-optimized
2022-03-04 17:01:10 +05:30
Pere Miquel Brull
e96ac838ff
Fix #3084 - Implement missing tests (#3117)
Fix #3084 - Implement missing tests
2022-03-04 06:59:47 +01:00
codingwithabhi
37457dc668
table-name-fail-fix- (#3114) 2022-03-03 13:44:34 -08:00
Ayush Shah
84fd717c57
Sql_alchemy_helper Engine fix (#3116)
* Added connect_args and options for creation of engine

* Added connect_args to profiler engine

* Error fixed on database_common profiler - isinstance error type or tuple of types
2022-03-03 23:39:06 +05:30
Mayur Singal
3f144a7370
fix #3077: Fixed validation error in ldap (#3079)
* fix #3077: fixed validation error

* removed print

* changes returntype to OMetaUserProfile
2022-03-03 11:11:07 +05:30
Pere Miquel Brull
71207de362
Fix #2875 - Profiler API Sink (#3011)
Fix #2875 - Profiler API Sink
2022-03-02 16:46:28 +01:00
Sriharsha Chintalapani
0b789788da
Fix #3051: Glossary-term suggest api has fqdn same as name (#3052) 2022-03-01 11:19:43 -08:00
Mayur Singal
98ddf5fba3
Fix #3041: Handled AttributeError (#3042) 2022-03-01 08:48:09 -08:00
Mayur Singal
c59a5e882d
fix #3035: handled none entity (#3036) 2022-03-01 08:06:16 -08:00
Alberto Miorin
fe5618c8f1
Fix #3037: metadata --version doesn't work (#3038) 2022-03-01 12:19:36 +01:00
Sriharsha Chintalapani
15d0440599
Fix #1220: catch and log any errors during the es index (#1220) 2022-03-01 12:27:50 +05:30
codingwithabhi
06d9329ae3
Atlas connector (#2855)
* Fix #1161: Add Atlas Connector

* Atlast Decimal type handled

* Atlas col_data_type handled

* Atlas columns exception handling implemented

* atlas-services-added-into-single-file

* import-updated

* authentication-fix

* atlas-connector-services-added

* code-smell-removed

* code-smell-removed

* code-smell-removed

* file-exist-check-added

* updated-mapping-json-logic

* pr-comment-changes-completed

* atlas-linegae-added

* atlas-linegae-added

* pr-changes-done

* code-formatted

* bug-resolved

* topic-bug-resolved

Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-03-01 11:50:14 +05:30
Ayush Shah
568845b65c
Power BI connector (#3019)
* Power BI Dashboard and Tiles added

* Added Powerbi dependency

* Modified Powerbi debug logs

* Bump up _version.py

* Modified Power BI - added failure status, resolved comments
2022-03-01 10:09:35 +05:30
Mayur Singal
c49af971a7
Fix #2984: added azure sso auth (#3002) 2022-02-28 11:44:28 -08:00
codingwithabhi
83c9b75a5a
Metabase lineage (#3003)
* metabase-lineage

* metabase-lineage-completed
2022-02-26 14:56:09 -08:00
Ayush Shah
412d61a875
Fix Okta Authentication and Validation - Ingestion (#2955) 2022-02-26 08:19:36 -08:00
Mayur Singal
e79be68bea
Fix #2937: added clickhouse usage connector (#2987) 2022-02-26 08:18:49 -08:00
Mayur Singal
b211435bac
Fix #2946: Presto Ingestion issues fix (#2986)
* Fix #2946: Presto Ingestion issues fix

* added col lenght

* removed print

* removed comment

* removed mysql import
2022-02-26 12:12:49 +05:30