3385 Commits

Author SHA1 Message Date
Pere Miquel Brull
1f11fdefb4
Databricks comments & view definitions in bulk (#10112)
* Add databricks comments

* Add view definition

* Add view definition
2023-02-06 08:55:15 +01:00
Suresh Srinivas
1990ac56d9
10041 Part1 - Refactor and clean up System APIs (#10042)
* 10041 Part1 - Refactor and clean up System APIs

* Update the config resource endpoint url
2023-02-03 16:25:40 -08:00
Onkar Ravgan
67479940cf
Use es for fqn in dbt datamodel (#10102)
* Use es for fqn in datamodel

* Corrected error message
2023-02-03 19:43:03 +05:30
Mayur Singal
01cad0f811
Add json zip support for datalake (#10100) 2023-02-03 13:03:55 +00:00
Mayur Singal
4d181ba8dc
Tableau Owner Fix (#10101) 2023-02-03 17:20:11 +05:30
Teddy
0a77f8a3f2
Fixes issue #10082 -- Unsupported field types for bigquery (#10091)
* fix(profile): Raise nicer error for BQ unsupported interval type for timestamp

* fix(profiler): run checkstyle
2023-02-02 19:00:56 +01:00
Teddy
5e1e975e0e
Fixes #10018 - Limit user fields to team (#10050)
* fix(dataInsight): Limit field for user request

* fix(dataInsight): revert changes added for testing
2023-02-02 15:00:55 +01:00
Nahuel
b9a3c06104
Bump main branch to version 1.0.0 (#10040)
* Bump to version 0.13.2

* Bump mvn projects to 1.0.0-SNAPSHOT

* Bump python projects to 1.0.0.dev0

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-02-02 12:56:14 +01:00
Teddy
4f707f8531
fix(pylint): adjust code to conform with pylint 2.16 release (#10055) 2023-02-01 18:03:56 +01:00
Pere Miquel Brull
f0f3f0be6a
Add looker unit tests (#9691)
* Add looker tests

* Empty-Commit

* Install GE for tests

* Fix usage details python name

* Add missing test requirement
2023-02-01 09:20:26 +00:00
Ayush Shah
747fcf569b
Add docs - quicksight, lineage... (#10023) 2023-01-31 15:17:40 +00:00
Teddy
ba08302ea1
Issue #7291 - Implements Table Rows Inserted to be Between test (#9813)
* staging commit

* staging commit

* refactor: partitioning logic

* refactor (tests): move to parametrized tests for test validations

* refactor: local variables into global

* (feat): Added logic for table row inserted test

* (feat): fix python checkstyle

* feature: extracted get_query_filter logic into its own function
2023-01-31 15:57:51 +01:00
Onkar Ravgan
b754121425
Added dbt Manifest V8 Compatibility (#10024)
* Added v8 compatibility

* Added condition
2023-01-31 14:06:06 +05:30
Teddy
0b3996d2fa
Remove profiler support for dynamodb (#10014)
* fix(DynamoDB): remove profiler support and cleaned up documentation

* fix(DynamoDB): remove supportProfiler filed in tests
2023-01-31 08:29:35 +01:00
Nahuel
10b15e782e
Fix: error getting 'certificatePath' from SSLConfig object (#10017) 2023-01-30 18:38:13 +00:00
Mayur Singal
8c7c21bd54
Fix Superset Owners & Descriptions (#10007) 2023-01-30 14:40:52 +00:00
Pere Miquel Brull
bd0e307912
Fix mlmodel client (#9997) 2023-01-30 11:41:39 +01:00
Ayush Shah
725542b607
Fix Glue PIpeline (#9991) 2023-01-30 06:13:01 +00:00
Onkar Ravgan
949989fb1c
Added dbt parser (#9982)
* Added dbt parser

* Added library dependency

* format and final fixes

* Addressed review comments

* Fixed typo
2023-01-29 20:47:39 +01:00
Pere Miquel Brull
2ec8b42e96
Get service connection info if not set (#9981) 2023-01-29 10:42:17 +01:00
Pere Miquel Brull
02c1f9ac94
Update view definition from Vertica dialect (#9980) 2023-01-29 13:28:43 +05:30
Pere Miquel Brull
47d814d059
Fix typo (#9969) 2023-01-28 18:39:59 +01:00
Pere Miquel Brull
d5c6c0212b
Fix e2e lineage count (#9964)
* Add vertica to the list

* test profiler

* Fix tests

* Fix tests

* Fix e2e lineage count
2023-01-27 15:33:03 +01:00
Nahuel
da1f911581
Doc: Add missing source config types in connectors config examples (#9955) 2023-01-27 15:30:48 +01:00
Pere Miquel Brull
f6d59f599e
Pin SQLAlchemy lower than 2 (#9952) 2023-01-27 15:26:30 +01:00
Pere Miquel Brull
192c992e21
Fix e2e tests when computing rows (#9958)
* Add vertica to the list

* test profiler

* Fix tests

* Fix tests
2023-01-27 12:55:57 +01:00
Noe Alejandro Perez Dominguez
d8674bd7f8
Deduplicate stl_querytext rows, fixes #8867 (#9870)
* Deduplicate stl_querytext rows, fixes #8867

* Update queries.py

Filter down result-set by joining with queries

* Reduce line character length
2023-01-27 10:36:49 +01:00
Pere Miquel Brull
4d474ab063
Add vertica e2e and fix checks (#9947)
* Add vertica e2e and fix checks

* Add missing task_id param
2023-01-27 10:22:12 +01:00
Nahuel
19f034eec4
Doc: Add Kafka troubleshooting section (#9941) 2023-01-26 18:52:06 +01:00
Mayur Singal
3f9364d8a4
Fix #9923: Fix usage for tables with same name (#9925)
* Fix #9923: Fix usage for tables with same name

* Fix #9923: Fix usage for tables with same name

* Update ingestion/src/metadata/ingestion/bulksink/metadata_usage.py

Co-authored-by: Nahuel <nahuel@getcollate.io>
2023-01-26 15:53:41 +00:00
Pere Miquel Brull
4650a453e1
Prepare Vertica Lineage and Usage (#9906)
* Prepare Vertica Lineage and Usage

* Simplify db usage

* Linting

* Revert postgres changes

* Revert postgres changes

* Add vertica flags
2023-01-26 14:34:33 +01:00
Pere Miquel Brull
c368116697
Add dbt cloud URL as an input parameter (#9911)
* Add dbt cloud URL as an input parameter

* support dbt cloud url from UI

* Flag dbtCloudUrl as required

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-01-26 09:36:03 +01:00
Nahuel
1de45aa2fb
Consider subprocess coverage in E2E CLI tests (#9935) 2023-01-26 07:55:54 +01:00
Abhishek Pandey
933e195ef6
salesforce-test-added (#9890)
* salesforce-test-added

* license-headers-added
2023-01-25 21:24:21 +05:30
Milan Bariya
49d48e0546
Update: Trino, Presto Logic and Doc (#9859)
* Update: Trino, Presto Logic and Doc

* Update: Trino, Presto Logic and Doc

* Update permissions in Doc

* Update openmetadata-docs/content/connectors/database/trino/index.md

Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>

Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2023-01-25 19:20:06 +05:30
Mayur Singal
dd82fad5c4
Fix sample data re-ingestion for upgrade test (#9905) 2023-01-25 17:44:18 +05:30
Nahuel
5608c2fde0
Add E2E CLI tests coverage to Sonar (#9908) 2023-01-25 11:51:25 +01:00
Milan Bariya
176111aa27
Added databricks in lineage model (#9887) 2023-01-25 16:06:48 +05:30
Mayur Singal
74d2132883
Supserset Connection Changes Doc (#9893) 2023-01-25 11:06:00 +05:30
Mayur Singal
b3f9e3ba97
Update snowflake Requirement Docs (#9900) 2023-01-25 11:05:14 +05:30
Pere Miquel Brull
bce8574f98
Dashboards and charts use the ID as name (#9810)
* Update dashboard names

* Fix: metabase lineage

Co-authored-by: Nihar Doshi <nihardoshi16@gmail.com>
2023-01-24 21:50:35 +05:30
Nahuel
fbc32bc873
Fix: MSSQL and DBT Redshift CLI E2E failing (#9899) 2023-01-24 16:04:42 +00:00
Nahuel
4274a145eb
Fix: change how CLI E2E are run (#9894) 2023-01-24 13:18:56 +00:00
NiharDoshi99
33090c149e
Fix: superset dummy credentials update (#9883) 2023-01-24 07:55:16 +00:00
Mayur Singal
d0191da8ec
Fix domo pipeline test connection (#9863) 2023-01-24 12:23:47 +05:30
Ayush Shah
d15d96ef1a
Fix Mapper Parser issue (#9865) 2023-01-24 11:21:50 +05:30
Pere Miquel Brull
f2fb0521c2
Update airflow loggers and rename ometa loggers (#9868)
* Update airflow loggers and rename ometa loggers

* ANSI print to logger

* Remove colored logging from tests

* Merge ometa_logger into the one used in loggers class

* linting

* linting

Co-authored-by: Nahuel Verdugo Revigliono <nahuel@getcollate.io>
2023-01-23 16:28:17 +01:00
Teddy
871cd0414e
Fix Partition for tables with profile sample + added mssql e2e tests (#9862)
* fix: partition for sampled data

* fix: added MSSQL E2E cli test

* fix: added secerets varibale to action file

* fix: python checkstyle
2023-01-23 14:16:44 +01:00
Mayur Singal
5145dced25
Datalake parquet files fix (#9860) 2023-01-23 17:44:34 +05:30
Nahuel
93057e476f
Fix: Superset encryption and decryption not working as expected (#9827)
* Fix superset sample data

* Update ingestion/examples/superset_data/service.json

* Update ingestion/examples/superset_data/service.json

* fix: pytest error (#9824)

* fix: pytest error

* fix: linting

* increased verbosity

* empty commit to re-run tests

* print registry and test definition set

* renamed columnValuesToBeUnique fqn

* removed print statements + verbosity

* Docs Using a Custom Ingestion Image in helm (#9835)

* Docs Using a Custom Ingestion Image in helm

* Update openmetadata-docs/content/deployment/kubernetes/index.md

Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>

* Update openmetadata-docs/content/deployment/kubernetes/index.md

Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>

* Update openmetadata-docs/content/deployment/kubernetes/index.md

* Update index.md

Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>

* fix(ui): ui changes for #9695 (#9770)

* Fix #9695: Teams show up in lower case

* changed owner advanced search filter field from name to display name

* fixed minor quick filter bug

* fixed failing cypress tests

* fixed failing unit tests

* Fixed failing cypress tests

* added comments in unit tests for better understanding of changes

* fixed failing cypress tests for advanced search

* improved logic for showing advanced search options

Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>

* Added powerbi docs for new field (#9836)

* Fix#9460: Avoid reuse inspector to get view definition (#9821)

* Avoid reuse inspector to get view definition

* Update openmetadata-sqllineage version

* Feat: Update GH action for E2E tests (#9839)

* Update E2E CI tests GH action

* Address PR comments

* Add converter for services when encypting or decrypting services objects

* Update E2E CI tests GH action

Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
2023-01-20 21:32:38 +00:00