Pere Miquel Brull
94d7500216
Fix #3248 & #3251 - Update metrics and column profile ( #3262 )
...
Fix #3248 & #3251 - Update metrics and column profile (#3262 )
2022-03-08 11:44:39 +01:00
Teddy
6ee31eb21f
Issue-2538: Add Iceberg type for Glue Table + Location Entity ( #3210 )
2022-03-07 08:27:14 -08:00
Pere Miquel Brull
bd7b91b448
Fix #3112 - col profile safety & sample data ( #3142 )
...
Fix #3112 - col profile safety & sample data (#3142 )
2022-03-04 13:14:11 +01:00
codingwithabhi
37457dc668
table-name-fail-fix- ( #3114 )
2022-03-03 13:44:34 -08:00
Mayur Singal
3f144a7370
fix #3077 : Fixed validation error in ldap ( #3079 )
...
* fix #3077 : fixed validation error
* removed print
* changes returntype to OMetaUserProfile
2022-03-03 11:11:07 +05:30
Pere Miquel Brull
71207de362
Fix #2875 - Profiler API Sink ( #3011 )
...
Fix #2875 - Profiler API Sink
2022-03-02 16:46:28 +01:00
Alberto Miorin
fe5618c8f1
Fix #3037 : metadata --version doesn't work ( #3038 )
2022-03-01 12:19:36 +01:00
codingwithabhi
06d9329ae3
Atlas connector ( #2855 )
...
* Fix #1161 : Add Atlas Connector
* Atlast Decimal type handled
* Atlas col_data_type handled
* Atlas columns exception handling implemented
* atlas-services-added-into-single-file
* import-updated
* authentication-fix
* atlas-connector-services-added
* code-smell-removed
* code-smell-removed
* code-smell-removed
* file-exist-check-added
* updated-mapping-json-logic
* pr-comment-changes-completed
* atlas-linegae-added
* atlas-linegae-added
* pr-changes-done
* code-formatted
* bug-resolved
* topic-bug-resolved
Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-03-01 11:50:14 +05:30
Ayush Shah
568845b65c
Power BI connector ( #3019 )
...
* Power BI Dashboard and Tiles added
* Added Powerbi dependency
* Modified Powerbi debug logs
* Bump up _version.py
* Modified Power BI - added failure status, resolved comments
2022-03-01 10:09:35 +05:30
Mayur Singal
c49af971a7
Fix #2984 : added azure sso auth ( #3002 )
2022-02-28 11:44:28 -08:00
Ayush Shah
412d61a875
Fix Okta Authentication and Validation - Ingestion ( #2955 )
2022-02-26 08:19:36 -08:00
Mayur Singal
e79be68bea
Fix #2937 : added clickhouse usage connector ( #2987 )
2022-02-26 08:18:49 -08:00
Pere Miquel Brull
4d928bab80
Fix #2965 - Additional Properties set to False ( #2976 )
2022-02-24 08:38:56 -08:00
Mayur Singal
4739aaa096
Fix #2905 : Added MSSQL Usage Connector ( #2948 )
...
* Fix #2509 : added mssql usage connector
* removed comment
* removed unused variables
* fixed code smell
* renamed sql variable
2022-02-24 01:01:03 +05:30
Ayush Shah
d2c64007cb
Table Constraints Added - Ingestion ( #2854 )
2022-02-19 09:16:15 -08:00
Sriharsha Chintalapani
891ff465a0
Fix #2838 : Fix airflow example DAG to import airflow libs ( #2839 )
2022-02-18 10:47:07 +05:30
Sriharsha Chintalapani
165f5bb0e9
Fix #2819 : Kafka connector security settings along with all configs to be retrieved ( #2823 )
...
* Fix #2811 : Log level option for running metadata
* Fix #2819 : Kafka connector security settings along with all configs to be retrieved
* Fix #2819 : Kafka connector security settings along with all configs to be retrieved
2022-02-17 15:17:43 +05:30
Mayur Singal
419448a28f
Issue 2727: Added Databricks Connector ( #2745 )
...
* Issue #2727 : Added Databricks Connector
* optional db and licence info
* make core_bump_version_dev
* host_port fix
* fixed service_type
* change package to sqlalchemy-databricks
* Update databaseService.json
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-02-17 11:22:13 +05:30
codingwithabhi
088dba3a1f
azuresql-config-added ( #2711 )
...
* azuresql-config-added
* rebased-with-main
* tables-ingestion-added
* duplicate-files-removed
* Update azuresql.py
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-02-17 09:23:07 +05:30
codingwithabhi
9011b27bf6
table-filter-added ( #2770 )
...
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-02-16 19:26:40 +05:30
Mayur Singal
5e4cd516bf
issue #2712 : Added SingleStore Connector ( #2747 )
2022-02-15 21:11:46 -08:00
Suresh Srinivas
28ba1a3c04
Fixes #2760 - Add entities for Glossary and initial API ( #2761 )
...
* Fixes #2760 - Add entities for Glossary and initial API
* Fixing merge issues
2022-02-15 20:54:46 -08:00
Mayur Singal
44cca4d020
fix #2663 : added table queries in sample data ( #2763 )
2022-02-15 15:45:58 +05:30
Mayur Singal
88afb17ee8
issue #2611 : added clickhouse connector ( #2679 )
...
Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
2022-02-10 08:32:01 -08:00
Ayush Shah
4f5f63b226
Issue-2672: Added support for personal access token name and secret and upgraded tableau version ( #2682 )
2022-02-08 23:02:56 +05:30
Ayush Shah
18d963c3d8
Issue-969: Superset Lineage Implemented ( #2659 )
2022-02-07 16:16:38 -08:00
codingwithabhi
66c3f43bbb
Fix #2537 : Ingestion Connector - DynamoDB
2022-02-07 10:52:11 -08:00
Mayur Singal
2d25da149d
Fix #2626 : add db2 connector ( #2661 )
...
Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
2022-02-07 10:49:59 -08:00
Ayush Shah
7013b070d2
Minor Data Type Display modification ( #2654 )
...
Characters modified from unicode for e.g. < to <
2022-02-07 18:07:08 +05:30
Pere Miquel Brull
7aceab41a2
Fix #2412 - Airflow Lineage Docs ( #2618 )
2022-02-06 12:51:05 -08:00
Ayush Shah
6f5a4eafe7
Bigquery support for env without passing any config ( #2160 )
...
* Bigquery support for env without passing any config
* Bigquery File formatted
2022-01-22 22:25:48 -08:00
Pere Miquel Brull
6693b4ae40
[issue-2262] - Simplify sample data DAG ( #2337 )
...
* Simplify sample data DAG
* Remove mkdir
* Generate sources before running compose
* Generate sources to install models to ingestion image
* Add python-on-whales for docker --start
* Remove python-on-whales from base
* Install venv
* Setup python
2022-01-22 16:37:56 +05:30
Sriharsha Chintalapani
10e71a7e09
Fix #2340 : Ingestion pipeline deployment refactor ( #2341 )
...
* Fix #2340 : Ingestion pipeline deployment refactor
2022-01-21 22:06:14 -08:00
codingwithabhi
ddf4ae261b
Bigquery credential configuration updated ( #2041 )
...
* bigquery-credentials-in-bigquery-json
* ui-changes-reverted
* credentials-moved-to-options
* Update bigquery.json
* Update bigquery.py
* delete-temp-code-added
* delete-temp-code-added
* removed-init-from-config
* Update datatypes_test.py
* Update column_helpers.py
* code-formatted
Co-authored-by: Abhishek <abhishek@Abhisheks-MacBook-Air.local>
Co-authored-by: = <=>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-01-21 17:23:02 +05:30
Matt
2a80caa8fe
Add logic to initialize relationships from seed data during application startup ( #2307 )
...
* Add logic to initialize relationships from seed data during application startup
* Remove ingestion related code for access control policies
* Move PolicyEvaluator init to PolicyResource
2022-01-20 22:18:03 -08:00
Sriharsha Chintalapani
0d3ded0742
Data Profiler Integration ( #2235 )
...
* Fix 2234: Data profiler integration
2022-01-18 20:25:43 -08:00
Matt
276128e8b0
Add MetadataOperations RBAC support for PATCH APIs ( #2128 )
...
* Add RBAC for PATCH APIs
* Expand scope to all resources except a few (Policy, User, Role, Team, resources that doesn't support PATCH)
* Fix code smells
2022-01-16 09:24:26 -08:00
Matt
f323afe0cd
Support role ingestion from sample data ( #2167 )
2022-01-11 16:51:18 -08:00
Sriharsha Chintalapani
cf6f438531
Fix #1994 : Add support for marking dataset entities as deleted ( #2141 )
...
* Fix #1994 : Add support for marking dataset entities as deleted
* Fix #1994 : Add support for marking dataset entities as deleted
* Fix #1994 : Add support for marking dataset entities as deleted
2022-01-11 11:42:53 +05:30
Alberto Miorin
c8de136cca
First part of #2091 : Python code in branch main is always formatted correctly ( #2093 )
2022-01-07 06:17:48 +01:00
Ayush Shah
19e5f737ff
Usage fix - removed service_name from bulk_sink ( #1956 )
...
* Query Usage fixed
* Usage updated
* Reformatted change
* Usage files reformatted
2022-01-04 17:12:10 +05:30
Ayush Shah
6175053dd4
Sample Topics fixed ( #2005 )
2022-01-03 11:00:04 -08:00
Matt
ab4c9ede25
Add ingestion for access_control_policies to support RBAC ( #1969 )
2021-12-30 08:25:09 -08:00
Matt
f7653f4122
Add support for GCS Lifecycle Policy Ingestion ( #1923 )
...
- Add example gcs.json
- Add support for MoveAction and DeleteAction
2021-12-26 20:26:33 -08:00
Matt
4588121520
Add support for DeleteAction in S3 Lifecycle Policy ( #1916 )
...
- Support Delete Action for S3
- Add Example s3.json
- Create AWSClient util
- Use AWSClient util in S3 ingestion source
- Remove ambiguity in policy filters by removing array with different types
2021-12-25 16:29:10 -08:00
Sriharsha Chintalapani
485fdea7b1
Fix #1854 : Support DBT 1.0 format ( #1855 )
...
* Fix #1854 : Support DBT 1.0 format
2021-12-20 12:25:26 -08:00
Sriharsha Chintalapani
3c254440a5
Fix #1841 : Presto Connector issues ( #1842 )
2021-12-18 23:27:51 -08:00
Pere Miquel Brull
2ec32017bb
[issue-1698] - Airflow DockerOperator setup ( #1730 )
...
* Fix typo
* Clean setup
* Update ingestion local image to be barebone on connector dependencies
* Prepare ingestion connectors base image
* Add system dependencies
* Prepare docker CLI
* Add docker provider
* Prepare entrypoint for the image
* Remove DBT pipeline as per Issue 1658
* Add TODO for ingestion build
* Bind docker socket
* Update comment
* Update README
* Use DockerOperator in sample data
* Build images with latest tag
* Prepare symlink to pass the volume to the DockerOperator
* Update README
* Prepare Base image for CI
* COPY multiple files into dir
* COPY multiple files into dir
* Remove DBT source as is now part of table ingestion
* Build docker base in run_local_docker
2021-12-18 16:41:38 +01:00
Sriharsha Chintalapani
df6232ef52
Fix #1808 : check for None in description ( #1809 )
2021-12-16 21:15:42 +01:00
Ayush Shah
d71c75aaa1
Update snowflake.py ( #1803 )
...
* Update snowflake.py
* snowflake needs database and warehouse
2021-12-16 10:46:49 -08:00
