yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-10-31 18:48:35 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,579 Commits 873 Branches 177 Tags
Commit Graph

391 Commits

Author SHA1 Message Date
Mayur Singal
0bea02c202
Superset Unit Tests (#10078) 2023-02-09 19:45:24 +05:30
Mayur Singal
392107bc4a
Datalake Avro & Json Lines Support (#10129) 2023-02-08 17:31:25 +00:00
NiharDoshi99
34a0cc147e
Fix: Added changes for Pii sensitive (#10119) 
* Fix: added changes for pii sensitive

* Fix: removed comments

* Fix: python checkstyle

* differtiate between sensitive and non sensitive tag

* fix: python test

* fix: added tests

* fix: maven CI
 2023-02-08 16:00:47 +00:00
NiharDoshi99
012f96a7a2
Domo issue (#10080) 
* Domo issue

* added pydantic models and changes as per comment

* Fix: added ownership details

* fixed pytest
 2023-02-07 17:16:26 +05:30
Pere Miquel Brull
fb15c896b3
Handle XLets in groups for AirflowLineageRunner (#10114) 
* Handle XLets in groups

* Linting

* Linting
 2023-02-07 06:49:46 +01:00
Pere Miquel Brull
f0f3f0be6a
Add looker unit tests (#9691) 
* Add looker tests

* Empty-Commit

* Install GE for tests

* Fix usage details python name

* Add missing test requirement
 2023-02-01 09:20:26 +00:00
Teddy
ba08302ea1
Issue #7291 - Implements Table Rows Inserted to be Between test (#9813) 
* staging commit

* staging commit

* refactor: partitioning logic

* refactor (tests): move to parametrized tests for test validations

* refactor: local variables into global

* (feat): Added logic for table row inserted test

* (feat): fix python checkstyle

* feature: extracted get_query_filter logic into its own function
 2023-01-31 15:57:51 +01:00
Teddy
0b3996d2fa
Remove profiler support for dynamodb (#10014) 
* fix(DynamoDB): remove profiler support and cleaned up documentation

* fix(DynamoDB): remove supportProfiler filed in tests
 2023-01-31 08:29:35 +01:00
Pere Miquel Brull
d5c6c0212b
Fix e2e lineage count (#9964) 
* Add vertica to the list

* test profiler

* Fix tests

* Fix tests

* Fix e2e lineage count
 2023-01-27 15:33:03 +01:00
Nahuel
da1f911581
Doc: Add missing source config types in connectors config examples (#9955) 2023-01-27 15:30:48 +01:00
Pere Miquel Brull
192c992e21
Fix e2e tests when computing rows (#9958) 
* Add vertica to the list

* test profiler

* Fix tests

* Fix tests
 2023-01-27 12:55:57 +01:00
Pere Miquel Brull
4d474ab063
Add vertica e2e and fix checks (#9947) 
* Add vertica e2e and fix checks

* Add missing task_id param
 2023-01-27 10:22:12 +01:00
Nahuel
1de45aa2fb
Consider subprocess coverage in E2E CLI tests (#9935) 2023-01-26 07:55:54 +01:00
Abhishek Pandey
933e195ef6
salesforce-test-added (#9890) 
* salesforce-test-added

* license-headers-added
 2023-01-25 21:24:21 +05:30
Pere Miquel Brull
bce8574f98
Dashboards and charts use the ID as name (#9810) 
* Update dashboard names

* Fix: metabase lineage

Co-authored-by: Nihar Doshi <nihardoshi16@gmail.com>
 2023-01-24 21:50:35 +05:30
Nahuel
fbc32bc873
Fix: MSSQL and DBT Redshift CLI E2E failing (#9899) 2023-01-24 16:04:42 +00:00
Nahuel
4274a145eb
Fix: change how CLI E2E are run (#9894) 2023-01-24 13:18:56 +00:00
Pere Miquel Brull
f2fb0521c2
Update airflow loggers and rename ometa loggers (#9868) 
* Update airflow loggers and rename ometa loggers

* ANSI print to logger

* Remove colored logging from tests

* Merge ometa_logger into the one used in loggers class

* linting

* linting

Co-authored-by: Nahuel Verdugo Revigliono <nahuel@getcollate.io>
 2023-01-23 16:28:17 +01:00
Teddy
871cd0414e
Fix Partition for tables with profile sample + added mssql e2e tests (#9862) 
* fix: partition for sampled data

* fix: added MSSQL E2E cli test

* fix: added secerets varibale to action file

* fix: python checkstyle
 2023-01-23 14:16:44 +01:00
Teddy
dcf220f867
fix: pytest error (#9824) 
* fix: pytest error

* fix: linting

* increased verbosity

* empty commit to re-run tests

* print registry and test definition set

* renamed columnValuesToBeUnique fqn

* removed print statements + verbosity
 2023-01-20 10:45:11 +01:00
Nahuel
ddff6e2875
Fix: Replace sqllineage with openmetadata-sqllineage (#9800) 
* Replace sqllineage with openmetadata-sqllineage

* Fix checkstyle and failing test

* Move logic to retrieve dialect of a service type into a class

* Improve py-check message when it fails

* Updated mapper

* Update code after merge
 2023-01-19 14:56:29 +01:00
Ayush Shah
14100d089f
Modify Charts (#9814) 2023-01-19 12:03:21 +00:00
Ayush Shah
02147b6502
Fix: Quicksight Dashboard and chart Metadata Ingestion (#9786) 
* Fix Quicksight json schema and errors

* Fix Quicksight lineage

* add pagination

* Generalize Pagination

* Fix Pylints

* update max results

* fix tests

* Add Pydantic Models for QuickSight resp

* Add fqn builder

* Fix Py lInt

* Fix Tests
 2023-01-18 20:37:41 +05:30
NiharDoshi99
f2649041f2
Fix: atlas test connection (#9791) 
* Fix: atlas test connection

* Fix: docs changes

* Fix: docs changes

* Fix: python checkstyle

* Fix: python test
 2023-01-18 20:16:07 +05:30
Milan Bariya
7f111584b8
Deprecate OMetaDatabaseAndTable (#9760) 
* Deprecate OMetaDatabaseAndTable

* Change Based On Comments

* Change Based On Comments

* Fix Pytest

* Fix Pytest
 2023-01-18 14:29:32 +01:00
Onkar Ravgan
043f1e0601
Dbt e2e test (#9767) 
* Added base files

* dbt e2e tests

* Added tag condition based on UI feedback

* Added owner and lineage tests

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2023-01-18 14:29:16 +01:00
Abhishek Pandey
86a57293ef
databricks-test-added (#9749) 
* databricks-test-added

* databricks-test-added
 2023-01-18 08:45:46 +01:00
Suresh Srinivas
ac157a8f0b
Fixes #9618 Allow apostrophe in entity names (#9642) 
* Fixes #9618 Allow apostrophe in entity names

* Add support for space in entity names (with exception) and column names

* Add regex checks to entity names, table column names to allow only required characters

* fix: allowed characters in entityName

* Add all the allowed special characters in the entity name in tests

* Allow '&', '/', ':' only for the table

* Override basic entityName by defining entity specific entityName for Python code gen

* fix: dagster migration + json schema path

* fix: python style check

* Fix test failure

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
 2023-01-17 11:34:29 -08:00
NiharDoshi99
2954f17832
Fix: Domo dashboard issue (#9743) 
* Fix: Domo dashboard issue

* Fix: added execption handling

* Fix: changed as per comment
 2023-01-17 21:27:15 +05:30
Ayush Shah
f11cdbfab1
Profile sample rows + Randomization of chunks for datalake (#9595) 2023-01-16 22:17:46 +05:30
NiharDoshi99
029dbe892e
Fix: added test case for atlas (#9678) 
* Fix: added test case for atlas

* Fix: resolved conflict

* Fix: changing back neo4j to old version

* Fix: changing back neo4j to old version

* Fix: changes as per comment

* Fix: changes as per comment

* Fix: python checkstyle
 2023-01-13 16:07:29 +05:30
Pere Miquel Brull
bf753a4dee
Fix #7768 - Update and organize versions (#9664) 
Fix #7768 - Update and organize versions (#9664)
 2023-01-11 07:05:12 +01:00
Pere Miquel Brull
84348d4748
Fix #8866 - bump datamodel-codegen (#9623) 
* Fix #8866 - bump datamodel-codegen

* Update connection options and arguments structure

* Add builders test

* Format

* Allow Any values in componentConfig

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
 2023-01-09 13:20:32 +01:00
NiharDoshi99
27c68fb208
test case for DomoDatabase (#9630) 
* test case for domo-database

* test case for domo-database

* Fix: pytest
 2023-01-06 12:02:01 +01:00
Mayur Singal
9ef32eef99
Fix #6829: Improve usage filtering (#9613) 2023-01-05 15:07:03 +00:00
Steve Russo
3a3f248edd
Issue 5592: Add Databricks Pipeline Connector (#9554) 
* feat: update databricks_client

* feat: create pipelineconnection & connection

* test: add unit test

* feat: add examples file

* fix: remove unused imports

* style: fix python style

* fix: code review adjustments

* fix: remove unneeded schema ref

* fix: format requirements

* fix: change context calls

* test: add pipeline_status test

* format: pylint format change

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2023-01-05 14:14:12 +01:00
Pere Miquel Brull
1f49ea46b5
Fix #9149 - Postgres foreign tables & reorganize queries for sources (#9588) 
Fix #9149 - Postgres foreign tables & reorganize queries for sources (#9588)
 2023-01-04 08:50:53 +01:00
8vicat
7e7528e13b
Improve SQL lineage parsing (#9537) 
* CTEs support
* Supports inline comments
* Expands `*` in sources
 2023-01-03 09:27:45 +01:00
NiharDoshi99
8f69386076
Fix: refactor get_column_type (#9559) 
* Fix: refactor get_column_type

* Fix: added changes as per comments

* Fix: pylint

* Fix: minor changes

* Fix: minor changes
 2023-01-03 10:28:38 +05:30
Pere Miquel Brull
7f21a7bced
Fix #8088 - Restructure source connections & clients (#9545) 2023-01-02 13:52:27 +01:00
Pere Miquel Brull
a4521fd664
Fix #6562 - Sources have their own package (#9521) 
Fix #6562 - Sources have their own package (#9521)
 2022-12-27 15:00:22 +01:00
Suresh Srinivas
758c976cba
Fixes #9259 Change Tags APIs to conform with rest of the APIs (#9260) 2022-12-26 12:32:17 -08:00
Pere Miquel Brull
ec6ebb3694
Organise dynamic importers (#9513) 
Organise dynamic importers (#9513)
 2022-12-23 12:17:06 +01:00
Teddy
9e01fe0636
feat(CLI): added migration logic cmd (#9437) 2022-12-21 07:55:18 +00:00
Mayur Singal
07aed10a7e
Snowflake E2E Tests (#8781) 2022-12-20 18:00:26 +05:30
Ayush Shah
2bf5eb9051
fix 7995: profileSample % and row number (#9104) 2022-12-20 14:55:11 +05:30
Milan Bariya
a3495c67f0
Databricks improve code and doc update (#9394) 
* Databricks improve code and doc update

* Change base on comments

* Fix pytest fail
 2022-12-19 18:53:10 +05:30
Pere Miquel Brull
3b7ae73473
Airflow e2e integration test (#9363) 
* Prep airflow operator integration tests

* Add integration test to Makefile
 2022-12-16 19:52:12 -08:00
Mayur Singal
5164a55341
Fix Mysql e2e test (#9364) 2022-12-16 19:39:02 -08:00
Teddy
3856d63f3a
Fixes #5522 by implementing timeout for the profiler (#9340) 
* feat(Profiler): Added timeout logic in the profiler workflow

* feat(Profiler): Added additional input in workflow UI

* fix(Profiler): timeout text blob in UI

* fix(Profiler): Return profile_results outside the context manager

* Fix(Profiler): Added localizer for timeout label and message

* Update ingestion/src/metadata/interfaces/sqalchemy/sqa_profiler_interface.py

Co-authored-by: Nahuel <nahuel@getcollate.io>

Co-authored-by: Nahuel <nahuel@getcollate.io>
 2022-12-16 17:01:12 +01:00