yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-10-31 10:39:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
10,698 Commits 866 Branches 177 Tags
Commit Graph

984 Commits

Author SHA1 Message Date
Teddy
0b3996d2fa
Remove profiler support for dynamodb (#10014) 
* fix(DynamoDB): remove profiler support and cleaned up documentation

* fix(DynamoDB): remove supportProfiler filed in tests
 2023-01-31 08:29:35 +01:00
Pere Miquel Brull
d5c6c0212b
Fix e2e lineage count (#9964) 
* Add vertica to the list

* test profiler

* Fix tests

* Fix tests

* Fix e2e lineage count
 2023-01-27 15:33:03 +01:00
Nahuel
da1f911581
Doc: Add missing source config types in connectors config examples (#9955) 2023-01-27 15:30:48 +01:00
Pere Miquel Brull
192c992e21
Fix e2e tests when computing rows (#9958) 
* Add vertica to the list

* test profiler

* Fix tests

* Fix tests
 2023-01-27 12:55:57 +01:00
Pere Miquel Brull
4d474ab063
Add vertica e2e and fix checks (#9947) 
* Add vertica e2e and fix checks

* Add missing task_id param
 2023-01-27 10:22:12 +01:00
Nahuel
1de45aa2fb
Consider subprocess coverage in E2E CLI tests (#9935) 2023-01-26 07:55:54 +01:00
Abhishek Pandey
933e195ef6
salesforce-test-added (#9890) 
* salesforce-test-added

* license-headers-added
 2023-01-25 21:24:21 +05:30
Pere Miquel Brull
bce8574f98
Dashboards and charts use the ID as name (#9810) 
* Update dashboard names

* Fix: metabase lineage

Co-authored-by: Nihar Doshi <nihardoshi16@gmail.com>
 2023-01-24 21:50:35 +05:30
Nahuel
fbc32bc873
Fix: MSSQL and DBT Redshift CLI E2E failing (#9899) 2023-01-24 16:04:42 +00:00
Nahuel
4274a145eb
Fix: change how CLI E2E are run (#9894) 2023-01-24 13:18:56 +00:00
Pere Miquel Brull
f2fb0521c2
Update airflow loggers and rename ometa loggers (#9868) 
* Update airflow loggers and rename ometa loggers

* ANSI print to logger

* Remove colored logging from tests

* Merge ometa_logger into the one used in loggers class

* linting

* linting

Co-authored-by: Nahuel Verdugo Revigliono <nahuel@getcollate.io>
 2023-01-23 16:28:17 +01:00
Teddy
871cd0414e
Fix Partition for tables with profile sample + added mssql e2e tests (#9862) 
* fix: partition for sampled data

* fix: added MSSQL E2E cli test

* fix: added secerets varibale to action file

* fix: python checkstyle
 2023-01-23 14:16:44 +01:00
Teddy
dcf220f867
fix: pytest error (#9824) 
* fix: pytest error

* fix: linting

* increased verbosity

* empty commit to re-run tests

* print registry and test definition set

* renamed columnValuesToBeUnique fqn

* removed print statements + verbosity
 2023-01-20 10:45:11 +01:00
Nahuel
ddff6e2875
Fix: Replace sqllineage with openmetadata-sqllineage (#9800) 
* Replace sqllineage with openmetadata-sqllineage

* Fix checkstyle and failing test

* Move logic to retrieve dialect of a service type into a class

* Improve py-check message when it fails

* Updated mapper

* Update code after merge
 2023-01-19 14:56:29 +01:00
Ayush Shah
14100d089f
Modify Charts (#9814) 2023-01-19 12:03:21 +00:00
Ayush Shah
02147b6502
Fix: Quicksight Dashboard and chart Metadata Ingestion (#9786) 
* Fix Quicksight json schema and errors

* Fix Quicksight lineage

* add pagination

* Generalize Pagination

* Fix Pylints

* update max results

* fix tests

* Add Pydantic Models for QuickSight resp

* Add fqn builder

* Fix Py lInt

* Fix Tests
 2023-01-18 20:37:41 +05:30
NiharDoshi99
f2649041f2
Fix: atlas test connection (#9791) 
* Fix: atlas test connection

* Fix: docs changes

* Fix: docs changes

* Fix: python checkstyle

* Fix: python test
 2023-01-18 20:16:07 +05:30
Milan Bariya
7f111584b8
Deprecate OMetaDatabaseAndTable (#9760) 
* Deprecate OMetaDatabaseAndTable

* Change Based On Comments

* Change Based On Comments

* Fix Pytest

* Fix Pytest
 2023-01-18 14:29:32 +01:00
Onkar Ravgan
043f1e0601
Dbt e2e test (#9767) 
* Added base files

* dbt e2e tests

* Added tag condition based on UI feedback

* Added owner and lineage tests

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2023-01-18 14:29:16 +01:00
Abhishek Pandey
86a57293ef
databricks-test-added (#9749) 
* databricks-test-added

* databricks-test-added
 2023-01-18 08:45:46 +01:00
Suresh Srinivas
ac157a8f0b
Fixes #9618 Allow apostrophe in entity names (#9642) 
* Fixes #9618 Allow apostrophe in entity names

* Add support for space in entity names (with exception) and column names

* Add regex checks to entity names, table column names to allow only required characters

* fix: allowed characters in entityName

* Add all the allowed special characters in the entity name in tests

* Allow '&', '/', ':' only for the table

* Override basic entityName by defining entity specific entityName for Python code gen

* fix: dagster migration + json schema path

* fix: python style check

* Fix test failure

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
 2023-01-17 11:34:29 -08:00
NiharDoshi99
2954f17832
Fix: Domo dashboard issue (#9743) 
* Fix: Domo dashboard issue

* Fix: added execption handling

* Fix: changed as per comment
 2023-01-17 21:27:15 +05:30
Ayush Shah
f11cdbfab1
Profile sample rows + Randomization of chunks for datalake (#9595) 2023-01-16 22:17:46 +05:30
NiharDoshi99
029dbe892e
Fix: added test case for atlas (#9678) 
* Fix: added test case for atlas

* Fix: resolved conflict

* Fix: changing back neo4j to old version

* Fix: changing back neo4j to old version

* Fix: changes as per comment

* Fix: changes as per comment

* Fix: python checkstyle
 2023-01-13 16:07:29 +05:30
Pere Miquel Brull
bf753a4dee
Fix #7768 - Update and organize versions (#9664) 
Fix #7768 - Update and organize versions (#9664)
 2023-01-11 07:05:12 +01:00
Pere Miquel Brull
84348d4748
Fix #8866 - bump datamodel-codegen (#9623) 
* Fix #8866 - bump datamodel-codegen

* Update connection options and arguments structure

* Add builders test

* Format

* Allow Any values in componentConfig

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
 2023-01-09 13:20:32 +01:00
NiharDoshi99
27c68fb208
test case for DomoDatabase (#9630) 
* test case for domo-database

* test case for domo-database

* Fix: pytest
 2023-01-06 12:02:01 +01:00
Mayur Singal
9ef32eef99
Fix #6829: Improve usage filtering (#9613) 2023-01-05 15:07:03 +00:00
Steve Russo
3a3f248edd
Issue 5592: Add Databricks Pipeline Connector (#9554) 
* feat: update databricks_client

* feat: create pipelineconnection & connection

* test: add unit test

* feat: add examples file

* fix: remove unused imports

* style: fix python style

* fix: code review adjustments

* fix: remove unneeded schema ref

* fix: format requirements

* fix: change context calls

* test: add pipeline_status test

* format: pylint format change

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2023-01-05 14:14:12 +01:00
Pere Miquel Brull
1f49ea46b5
Fix #9149 - Postgres foreign tables & reorganize queries for sources (#9588) 
Fix #9149 - Postgres foreign tables & reorganize queries for sources (#9588)
 2023-01-04 08:50:53 +01:00
8vicat
7e7528e13b
Improve SQL lineage parsing (#9537) 
* CTEs support
* Supports inline comments
* Expands `*` in sources
 2023-01-03 09:27:45 +01:00
NiharDoshi99
8f69386076
Fix: refactor get_column_type (#9559) 
* Fix: refactor get_column_type

* Fix: added changes as per comments

* Fix: pylint

* Fix: minor changes

* Fix: minor changes
 2023-01-03 10:28:38 +05:30
Pere Miquel Brull
7f21a7bced
Fix #8088 - Restructure source connections & clients (#9545) 2023-01-02 13:52:27 +01:00
Pere Miquel Brull
a4521fd664
Fix #6562 - Sources have their own package (#9521) 
Fix #6562 - Sources have their own package (#9521)
 2022-12-27 15:00:22 +01:00
Suresh Srinivas
758c976cba
Fixes #9259 Change Tags APIs to conform with rest of the APIs (#9260) 2022-12-26 12:32:17 -08:00
Pere Miquel Brull
ec6ebb3694
Organise dynamic importers (#9513) 
Organise dynamic importers (#9513)
 2022-12-23 12:17:06 +01:00
Teddy
9e01fe0636
feat(CLI): added migration logic cmd (#9437) 2022-12-21 07:55:18 +00:00
Mayur Singal
07aed10a7e
Snowflake E2E Tests (#8781) 2022-12-20 18:00:26 +05:30
Ayush Shah
2bf5eb9051
fix 7995: profileSample % and row number (#9104) 2022-12-20 14:55:11 +05:30
Milan Bariya
a3495c67f0
Databricks improve code and doc update (#9394) 
* Databricks improve code and doc update

* Change base on comments

* Fix pytest fail
 2022-12-19 18:53:10 +05:30
Pere Miquel Brull
3b7ae73473
Airflow e2e integration test (#9363) 
* Prep airflow operator integration tests

* Add integration test to Makefile
 2022-12-16 19:52:12 -08:00
Mayur Singal
5164a55341
Fix Mysql e2e test (#9364) 2022-12-16 19:39:02 -08:00
Teddy
3856d63f3a
Fixes #5522 by implementing timeout for the profiler (#9340) 
* feat(Profiler): Added timeout logic in the profiler workflow

* feat(Profiler): Added additional input in workflow UI

* fix(Profiler): timeout text blob in UI

* fix(Profiler): Return profile_results outside the context manager

* Fix(Profiler): Added localizer for timeout label and message

* Update ingestion/src/metadata/interfaces/sqalchemy/sqa_profiler_interface.py

Co-authored-by: Nahuel <nahuel@getcollate.io>

Co-authored-by: Nahuel <nahuel@getcollate.io>
 2022-12-16 17:01:12 +01:00
NiharDoshi99
3190ac2516
Fix: making token as optional field (#9314) 2022-12-16 14:17:47 +00:00
Nahuel
819001182f
Fix#9251: DB2 connection config and ingestion update (#9322) 
* DB2 connection config and ingestion update

* Update ingestion/src/metadata/ingestion/source/database/common_db_source.py

Co-authored-by: Ayush Shah <ayush@getcollate.io>

* Update ingestion/src/metadata/ingestion/source/database/common_db_source.py

Co-authored-by: Ayush Shah <ayush@getcollate.io>

* Update bootstrap/sql/com.mysql.cj.jdbc.Driver/v007__create_db_connection_info.sql

Co-authored-by: Ayush Shah <ayush@getcollate.io>
 2022-12-16 07:43:18 +01:00
Onkar Ravgan
b539b299ee
Integrated schema parsers (#9305) 
* Integrated schema parsers

* Addressed review comments

* fixed pytests
 2022-12-15 16:54:55 +05:30
Ayush Shah
a6ae9fd11a
Add Test Suite Implementation for Datalake (#9235) 2022-12-14 21:14:51 +05:30
Teddy
d1a739ec55
Fixes #9025 -- Added deletion of WebAnalytics events in dataInsight Workflow (#9208) 2022-12-13 11:43:29 +01:00
Nahuel
9a4e3a7a46
Fix#8424: Remove brackets from tables and schemas on lineage (#9257) 
* Refactor LineageRunner use

* Address PR comments

* Address pylint errors

* Fix failing test

* Remove brackets from tables and schemas on lineage
 2022-12-13 06:40:37 +01:00
Pere Miquel Brull
c75ba751b7
Fix #9116 & #8284 - Clean tableau source, fix ownership, add description and SSL verification (#9241) 
Fix #9116 & #8284 - Clean tableau source, fix ownership, add description and SSL verification (#9241)
 2022-12-13 06:36:55 +01:00