Pere Miquel Brull
1fb0e7c489
Fix #2878 & #2877 - Implement Metrics and Validate Composed Metrics ( #2926 )
...
Fix #2878 & #2877 - Implement Metrics and Validate Composed Metrics
2022-02-24 07:08:39 +01:00
Mayur Singal
4739aaa096
Fix #2905 : Added MSSQL Usage Connector ( #2948 )
...
* Fix #2509 : added mssql usage connector
* removed comment
* removed unused variables
* fixed code smell
* renamed sql variable
2022-02-24 01:01:03 +05:30
Ayush Shah
fab36fe1b2
fix 2918: Added Error Handling in policy tags ( #2933 )
2022-02-24 00:56:14 +05:30
Mayur Singal
f9182a6244
Fix #2901 : fixed error message ( #2917 )
...
* Fix #2901 : fixed error message
* moved error message to helper.py
* replaced error to http
2022-02-23 00:20:36 +05:30
Pere Miquel Brull
1224d20a36
Fix #2894 - Profiler Processor & Metrics ( #2900 )
...
Fix #2894 - Profiler Processor & Metrics (#2900 )
2022-02-22 08:09:02 +01:00
Pere Miquel Brull
1656ca460c
Check for empty payload ( #2903 )
2022-02-21 14:19:27 -08:00
Mayur Singal
1e556d2ac1
fix #2869 : snowflake-usage processor fixed ( #2896 )
2022-02-21 09:05:24 -08:00
Pere Miquel Brull
e55579aaa8
Fix #2845 - Init Profiler Workflow ( #2862 )
...
* Fix list typing
* Add sqlite service
* Add sqlite service
* Add sqlite service
* Refactor validation into class
* Refactor validation into class
* Prepare table simple profiler
* Add note
* test ORM conversion
* Prepare workflow config utilities
* Prepare workflow skeleton
* Use new core Validation
* Refactor workflow config parsing
* Add comment
* Simplify workflow validations
* Fix table metric check
* Add init for convenience, otherwise interpreter cries when trying to __call__ the get result
* Fix table metric check
* Format
* Format
* Fix table list and metrics init
* Prepare profiler workflow integration tests
* Bump version
* Fix pycharm imports
* format
2022-02-20 17:55:12 +01:00
Ayush Shah
d2c64007cb
Table Constraints Added - Ingestion ( #2854 )
2022-02-19 09:16:15 -08:00
Ayush Shah
0e34028d05
Tableau Lineage Implementation ( #2850 )
...
* Tableau Lineage Implementation
* Handling Code Smell
2022-02-19 09:15:35 -08:00
Ayush Shah
959d441661
Glossary Entity Added - Ingestion fix ( #2794 )
...
* Added Glossary
* Modified Glossary: Mixin and suffix
* Glossary Mixin Methods
* Sample Data and Mixin for Glossary
* Modified comments
* Glossary Mixin Updated
* Added Glossary import
* Import issues fixed, removed upper limit for jinja2 as it conflicted
* Removed Glossary Import
2022-02-18 12:50:55 +05:30
Pere Miquel Brull
f304d290b4
Fix #2751 - Init ORM Profiler ( #2831 )
...
* ORM Profiler skeleton
* Fix table name within service
* Add license
* Prepare custom types
* Fix converter
* Compute stddev only on numeric
* Prepare smart registries
* Update tests
* Update results retrieval
* Fix composed metrics result
* Format
* Add missing type
* Add _label decorator
* clean readme
* clean readme
* Filter types when profiler runs not allowed metric types
* Fix null ratio
* Add proper type
* RuleMetric skeleton
* Prepare table metrics
* Update simple profiler
* Format
* Define test expression grammar and node visiting
* Unify metric registry
* Prepare validation core
* Add grammar lib
* Add safe get
* Format
* Allow decimals in grammar
* Test validation conversion
* Fix validation conversion and test
* Rename to row_number
* Update READMEs
* Format
* Row number naming
* Fix rename
2022-02-18 07:48:38 +01:00
Sriharsha Chintalapani
891ff465a0
Fix #2838 : Fix airflow example DAG to import airflow libs ( #2839 )
2022-02-18 10:47:07 +05:30
Igor Kramer
316e02cdfd
FIX #2802 : Skip partitions and add datatypes ( #2803 )
...
* ISSUE-2802: Skip partitions and add datatypes
* ISSUE-2802: formatting changes
2022-02-17 10:09:09 -08:00
Ayush Shah
c79f1f286d
Added Users to TableQueries ( #2826 )
...
* Added Users to TableQueries
* User EntityReference
* User EntityReference modification
2022-02-17 20:46:14 +05:30
Sriharsha Chintalapani
165f5bb0e9
Fix #2819 : Kafka connector security settings along with all configs to be retrieved ( #2823 )
...
* Fix #2811 : Log level option for running metadata
* Fix #2819 : Kafka connector security settings along with all configs to be retrieved
* Fix #2819 : Kafka connector security settings along with all configs to be retrieved
2022-02-17 15:17:43 +05:30
Mayur Singal
419448a28f
Issue 2727: Added Databricks Connector ( #2745 )
...
* Issue #2727 : Added Databricks Connector
* optional db and licence info
* make core_bump_version_dev
* host_port fix
* fixed service_type
* change package to sqlalchemy-databricks
* Update databaseService.json
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-02-17 11:22:13 +05:30
Mayur Singal
dfebc85c4e
fix #2790 : fixed usage ingestion issue ( #2805 )
2022-02-17 10:46:23 +05:30
Sriharsha Chintalapani
50799227e4
Fix #2817 : Remove indexing of Glossary as another Elasticsearch index ( #2818 )
2022-02-16 20:10:26 -08:00
codingwithabhi
088dba3a1f
azuresql-config-added ( #2711 )
...
* azuresql-config-added
* rebased-with-main
* tables-ingestion-added
* duplicate-files-removed
* Update azuresql.py
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-02-17 09:23:07 +05:30
codingwithabhi
9011b27bf6
table-filter-added ( #2770 )
...
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-02-16 19:26:40 +05:30
Ayush Shah
4aee82bfd8
Fix for #2597 - added check with column type mapping ( #2598 )
...
* Fix for #2597 - added check with column type mapping
* Optimized Column Type Parsing
* Optimized Column Type Parsing
* SQL Source Bigquery Policy Tags check added
2022-02-16 11:05:12 +05:30
codingwithabhi
f1c62a70b7
service-type-added ( #2769 )
2022-02-16 10:56:40 +05:30
Sriharsha Chintalapani
0ccba07059
Minor Fix: elastic search mappings ( #2772 )
2022-02-16 10:44:11 +05:30
Mayur Singal
78f5b51e63
fix #2663 : added table queries put request ( #2768 )
2022-02-15 21:12:38 -08:00
Mayur Singal
5e4cd516bf
issue #2712 : Added SingleStore Connector ( #2747 )
2022-02-15 21:11:46 -08:00
Suresh Srinivas
28ba1a3c04
Fixes #2760 - Add entities for Glossary and initial API ( #2761 )
...
* Fixes #2760 - Add entities for Glossary and initial API
* Fixing merge issues
2022-02-15 20:54:46 -08:00
Mayur Singal
44cca4d020
fix #2663 : added table queries in sample data ( #2763 )
2022-02-15 15:45:58 +05:30
Pere Miquel Brull
76f4ccd590
Fix #2735 - Simplify Airflow properties extraction ( #2749 )
...
* Simplify lineage properties extraction
* Add network name
* Format
2022-02-14 16:53:42 +01:00
Sriharsha Chintalapani
562d6b39ef
Fix #2704 : Add suggestions API to query users and teams like autocomplete ( #2742 )
2022-02-14 07:47:07 -08:00
Pere Miquel Brull
1b2ea54d4f
Fix #2316 - Append Tasks & Add Status in Airflow ( #2738 )
...
* Add license
* Add date to timestamp helper
* Prepare pipeline status operation
* Update and clean tasks from client
* Update tasks from client
* Check if pipeline is empty
* Keep all current pipeline info
* Append and clean tasks
* format
* Add status information
* Test pipelineStatus update
* Update task on clear
* Log status on callback
* Update lineage and status docs
* Update lineage docs
* Format
* Logic to handle DAG status
* Lint and format
* Update lineage tests
2022-02-13 17:51:25 +01:00
Mayur Singal
06ed718235
Fix #2715 : Added clickhouse datatypes ( #2732 )
2022-02-12 10:03:03 -08:00
Ayush Shah
f189c760bc
Ordinal Position modified ( #2733 )
2022-02-12 10:02:47 -08:00
codingwithabhi
b51c0d92b1
added-named-parameter ( #2680 )
...
* added-named-parameter
* metadata-config-fix-and-connector-tested
2022-02-11 07:53:46 -08:00
Igor Kramer
6891cd1e62
ISSUE-2716 Skip lineage and empty charts ( #2717 )
2022-02-11 10:06:31 +05:30
Mayur Singal
88afb17ee8
issue #2611 : added clickhouse connector ( #2679 )
...
Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
2022-02-10 08:32:01 -08:00
Igor Kramer
713375175d
ISSUE #2706 : Fix tableau ingestion bugs ( #2707 )
...
* ISSUE-2706: Fix tableau ingestion bugs
2022-02-10 08:23:53 -08:00
Mayur Singal
c64f3fef34
Fix #2678 : Fixed vertica import errors and invalid dataTypeDisplay ( #2709 )
...
Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
2022-02-10 07:22:29 -08:00
Mayur Singal
4abede9f9d
Fix #2585 : dataprofile module perform check on first object only ( #2671 )
...
* Fix #2585 : dataprofile module perform checks on first object only
* py format
Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
2022-02-09 11:29:12 +05:30
Ayush Shah
4f5f63b226
Issue-2672: Added support for personal access token name and secret and upgraded tableau version ( #2682 )
2022-02-08 23:02:56 +05:30
codingwithabhi
a4f3999904
Bigquery-Usage-ADC ( #2662 )
...
* creds-code-added
* creds-logic-updated
* implemented-close()-method
* temp-files-error-fix
* removed-ad-hoc-tag-implementations-from-ometa_api
2022-02-08 17:59:56 +05:30
Ayush Shah
18d963c3d8
Issue-969: Superset Lineage Implemented ( #2659 )
2022-02-07 16:16:38 -08:00
codingwithabhi
66c3f43bbb
Fix #2537 : Ingestion Connector - DynamoDB
2022-02-07 10:52:11 -08:00
Mayur Singal
2d25da149d
Fix #2626 : add db2 connector ( #2661 )
...
Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
2022-02-07 10:49:59 -08:00
Ayush Shah
0e62db641d
Added check for policy tags ( #2655 )
2022-02-07 10:49:32 -08:00
Mayur Singal
b4ebf5ce87
Fix #2591 : added get_view_definition for vertica ( #2650 )
...
Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
2022-02-07 19:19:44 +05:30
Ayush Shah
7013b070d2
Minor Data Type Display modification ( #2654 )
...
Characters modified from unicode for e.g. < to <
2022-02-07 18:07:08 +05:30
codingwithabhi
888e225544
Snowflake ingestion fix ( #2622 )
...
* creds-removed
* fixed-ingestion-fail-and-variant-type-error
2022-02-07 10:17:21 +05:30
Pere Miquel Brull
08a46814e6
Fix #2625 : Use pytest 7.0.0 ( #2640 )
2022-02-06 14:41:56 -08:00
Pere Miquel Brull
7aceab41a2
Fix #2412 - Airflow Lineage Docs ( #2618 )
2022-02-06 12:51:05 -08:00
