yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-07-05 16:19:35 +00:00
Code Issues Packages Projects Releases Wiki Activity
12,962 Commits 654 Branches 142 Tags
Commit Graph

1157 Commits

Author SHA1 Message Date
Abhishek Pandey
bef4d412ce
uniontype-coverage-added (#6348) 2022-07-26 15:21:34 +02:00
Teddy
aae4410c93
Fies #6183 - Ability to set profile sample at the profilier workflow level (#6292) 
Fies  #6183 - Ability to set profile sample at the profilier workflow level (#6292)
 2022-07-25 12:08:20 +02:00
Pere Miquel Brull
7560635114
Fix #6280 - Bump DeltaLake version, tests and docs (#6307) 
Fix #6280 - Bump DeltaLake version, tests and docs (#6307)
 2022-07-24 18:49:15 +02:00
Pere Miquel Brull
810e6c7dfc
Fix #5822 - Remove looker env and rename props (#5700) 
Fix #5822 - Remove looker env and rename props (#5700)
 2022-07-24 16:02:26 +02:00
Francisco J. Jurado Moreno
78dd44ed32
[TASK-6140] Add OMeta header to SQLAlchemy-based queries (#6293) 
[TASK-6140] Add OMeta header to SQLAlchemy-based queries (#6293)
 2022-07-23 12:31:26 +02:00
Francisco J. Jurado Moreno
756dae5605
[TASK-6241] Cache ES ingestion when ingesting usage (#6276) 
* Cache ingestion

* Uncomment secrets manager

* Fix experiment stuff

* Fix style

* Add LRU cache

* Add tests

* Fix code smell
 2022-07-22 23:33:22 +02:00
Nahuel
4d4a2fc2cf
FIX#5916: Only admin user retrieves connection params when external Secrets Manager is configured (#6228) 
* Implementation for stop sending connection credentials when user is BOT

* Change way we add the connection to the service in the Secret Manager

* Services connection is not required as we want to stop returning it when SM is configured
 2022-07-22 14:02:29 -07:00
Teddy
28623c7455
Fixes #6078 - DB column named metadata conflict with SQA reserved attribute (#6282) 2022-07-22 19:05:32 +02:00
Mayur Singal
75ccc803ad
Fix #5410: Topology for Pipeline Services (#6085) 
Fix #5410: Topology for Pipeline Services (#6085)
 2022-07-22 13:59:14 +02:00
Teddy
893665f33d
Fixes #6031 & #6236 -- Handle binary field & UUID when ingesting sample data (#6233) 
Fixes #6031 & #6236 -- Handle binary field & UUID when ingesting sample data (#6233)
 2022-07-21 20:14:44 +02:00
Teddy
e1fac99353
Fixes #5723 and implement interface processor logic (#6219) 
* Added datetime for min/max

* Added profiler interface

* Update core.py to work with profiler_interface

* Implement interface logic for orm_profiler object

* Fix unique_ratio logic

* removed changes to table.json

* Added Protocol for type hint

* Changed protocol to abc + fixed sonar code smell

* Fixed py_format
 2022-07-20 17:54:10 +02:00
Pere Miquel Brull
2936c0e4e3
Remove retry in ES query (#6191) 
Remove retry in ES query (#6191)
 2022-07-20 12:44:50 +02:00
Nahuel
0602e8c894
Fix#5924: Implement secret manager for Airflow REST Plugin (#6128) 
Fix#5924: Implement secret manager for Airflow REST Plugin (#6128)
 2022-07-19 14:51:44 +02:00
Pere Miquel Brull
9fa5d5cab2
Fix #6154 - Python Client to PATCH descriptions (#6165) 
* Prepare patch description utilities for python client

* Improvements
 2022-07-19 12:58:58 +02:00
Pere Miquel Brull
7b6332edef
Fix #6045 - Parse gracefully the connection on Test Connection (#6149) 
* Parse gracefully test connection

* Test file renames
 2022-07-18 18:50:27 +02:00
Pere Miquel Brull
479a8de486
Fix #5885 - Provider loading fails for airflow <2.3 (#5927) 
Fix #5885 - Provider loading fails for airflow <2.3 (#5927)
 2022-07-14 15:07:39 +02:00
Milan Bariya
954635b6dc
FIX 5907: Ingest Trino complex type (#5931) 
* FIX 5907: Ingest Trino complex type

* Included Row & Array Datatype

* Resolved review comments

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
 2022-07-14 11:28:50 +05:30
nna077
baa5295cc2
Add Date/Time Metrics In Profiler Tab (#5821) 
Add Date/Time Metrics In Profiler Tab (#5821)
 2022-07-13 21:23:03 +02:00
Teddy
d097199d2f
Added validation in profiler workflow to ensure service name exists and raise more explicit error (#6036) 
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2022-07-13 14:43:48 +02:00
Teddy
f0926d9b75
Fixes #3463 -- Implement Custom SQL Data Quality Tests (#5973) 
Fixes #3463 -- Implement Custom SQL Data Quality Tests (#5973)
 2022-07-12 06:21:11 +02:00
Mayur Singal
ebc9a24ee0
Connection options fix (#5971) 
* Connection Options Fix

* Fixed Condition
 2022-07-08 17:44:18 +02:00
Teddy
11487571b0
Fixes #5964 -- Trino Median function and columnValuesToMatchRegex wording (#5965) 
* added logic for trino median function

* Fixed columnValuesToMatchRegex result wording

* Replaced table with column for toMatchRegex test
 2022-07-08 15:55:50 +02:00
Teddy
48f6553fb3
Fixes #5866 -- Implement Support for Median Metric (#5887) 
* Added additional table + test coverage

* Added logic for front end input fields

* Added comment for median metric

* skipping `Update owner and check description` cypress test

* Added support to run window metrics for the profiler

* Fix except code smell

* moved median metric to windown folder

* Fix pyformat

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
 2022-07-07 06:58:42 +02:00
Teddy
3a7c11424b
Fixes #3133 -- Adding Additional Column Tests (#5867) 
* Added additional table + test coverage

* Added logic for front end input fields

* Added comment for median metric

* skipping `Update owner and check description` cypress test

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
 2022-07-06 10:12:29 +02:00
Teddy
e026d625d6
Fixes #5713 where profileSample default to 100 (#5714) 
* Fix random sample modulo and rand for postgres

* Remove sampleProfile default and return declarative meta when no sample profile is passed

* fixed py_format

* isolated random sample partition logic into its own methid
 2022-06-28 19:27:55 +02:00
Mayur Singal
71bf32e438
Fix #5562: Handle Analysis Date for Usage (#5582) 
Fix #5562: Handle Analysis Date for Usage (#5582)
 2022-06-27 15:16:12 +02:00
Milan Bariya
338d1efe40
FIX 5419: Glue pipeline connector not working (#5467) 
FIX 5419: Glue pipeline connector not working (#5467)
 2022-06-27 11:38:41 +02:00
Sriharsha Chintalapani
ad0c6dad88
Fix #5613: Add fernet encryption and remove connection details for non admin users (#5614) 
* Fix #5613: Add fernet encryption and remove connection details for non-admin users
 2022-06-24 16:44:37 -07:00
Teddy
7431c1c226
[Issue-5188] Implement User Query for Sampler and Profiler (#5578) 
* Added custom query sample for sample data ingestion

* Added logic to run table profiling against user's query

* Added tests for user query logic in profiler and sampler

* Added user profiling to tableProfile + fixed format

* staging commit

* Added logic to add profileQuery to table entity

* Added limit to sample rows
 2022-06-24 14:46:34 +02:00
Pere Miquel Brull
0144a1d46a
Fix #5423 & #3412 - Looker Lineage and Usage (#5581) 
Fix #5423 & #3412 - Looker Lineage and Usage (#5581)
 2022-06-23 13:01:40 +02:00
Pere Miquel Brull
0ecc9f0da6
Fix #5459 - Remove sql-metadata in favor of sqllineage (#5494) 
Fix #5459 - Remove sql-metadata in favor of sqllineage (#5494)
 2022-06-21 18:02:50 +02:00
Pere Miquel Brull
a14de6eee1
Fix #5367 - MlFlow connector & MLServices (#5446) 
Fix #5367 - MlFlow connector & MLServices (#5446)
 2022-06-21 14:54:36 +02:00
Teddy
5067e24374
[ISSUE-4723] Fix Snowflake Case Sensitive Error with Profiler (#5533) 
* Fixed snowflake profiler + enabled profiler integration tests

* Fixed py formating
 2022-06-20 22:23:17 +02:00
Teddy
c151a34c70
[ISSUE-4659] Add Additional GE Tests to the Profiler (#5506) 
* Added table tests 'column count to be between' and 'column name to be in set'

* Added 'column values to be in set' test

* Fixed wrong filed name when invoking TestCaseResult

* Added col min and col max to be between

* Fix formatting

* Fix test after adding column name list to table profile

* Added 'column values sum between' test

* Added 'table column name to exist' test

* Fixed JSON description white space + spelling mistake
 2022-06-20 08:54:42 +02:00
Teddy
28336b0a43
Fix #3575 Data Quality - Partitioned Tables (#5441) 
Co-authored-by: Teddy Crepineau <teddycrepineau@Teddys-MacBook-Pro.local>
Co-authored-by: Vivek Ratnavel Subramanian <vivekratnavel90@gmail.com>
 2022-06-14 12:37:44 -07:00
Pere Miquel Brull
02fece727b
Fix #5307 - Define a better topology for Ingestion Sources (#5366) 
Fix #5307 - Define a better topology for Ingestion Sources (#5366)
 2022-06-13 12:17:54 +02:00
Snyk bot
c717cb1545
fix: ingestion/tests/integration/lineage/airflow/requirements.txt to reduce vulnerabilities (#5178) 
The following vulnerabilities are fixed by pinning transitive dependencies:
- https://snyk.io/vuln/SNYK-PYTHON-FLASKAPPBUILDER-2433100
 2022-06-08 13:36:31 -07:00
Pere Miquel Brull
8e9d0a73f6
Fix #3573 - Sample Data refactor & ORM converter improvements (#5265) 
Fix #3573 - Sample Data refactor & ORM converter improvements (#5265)
 2022-06-08 16:10:40 +02:00
Milan Bariya
ec1aca926a
Fix #5210: Review database field for connectors (#5225) 
* Fix #5210: Review database field for connectors
 2022-06-06 10:48:23 +05:30
Pere Miquel Brull
9138c70dee
Fix #4696 - Only store URL suffixes when extracting metadata (#5281) 
* Only store URL suffix

* Fix test service name
 2022-06-03 11:43:40 +02:00
Mayur Singal
b3428771a3
Fix #4767: Refactor Usage & Lineage (#5190) 
Fix #4767: Refactor Usage & Lineage (#5190)
 2022-06-03 10:12:28 +02:00
Onkar Ravgan
4671ca00b0
Refactored PowerBi Connector (#5107) 
Refactored PowerBi Connector (#5107)
 2022-06-01 10:36:12 +02:00
Abhishek Pandey
50409eac24
removed-required-from-aws-credential (#5219) 
removed-required-from-aws-credential (#5219)
 2022-05-31 08:45:02 +02:00
Abhishek Pandey
12e8a1fcf6
required-fields-updated-in-snowflake-and-athena (#5143) 
required-fields-updated-in-snowflake-and-athena (#5143)
 2022-05-30 14:57:26 +02:00
Pere Miquel Brull
0005bc1292
Fix #5108 - Add Map and Struct support for Athena & Block them for profiler (#5181) 
Fix #5108 - Add Map and Struct support for Athena & Block them for profiler (#5181)
 2022-05-30 06:53:16 +02:00
Sriharsha Chintalapani
cf3d4314dd
Fix #1607 Operational metrics for OpenMetadata system observability (#4386) 
Co-authored-by: Akash-Jain <Akash.J@deuexsolutions.com>
 2022-05-27 15:39:13 +05:30
Pere Miquel Brull
02c2dd9494
Fix #4689 - Rename FQDN to FQN and clean unused code (#5165) 2022-05-26 21:00:18 +02:00
Teddy
bbe5cfa9e7
[ISSUE-3450] openmetadata-ingestion great-expectations submodule (#4752) 
[ISSUE-3450] openmetadata-ingestion great-expectations submodule (#4752)
 2022-05-26 15:16:03 +02:00
Mayur Singal
f9bb54ed91
Fix #5162: Sample usage fixed (#5163) 
* Fix #5162: Sample usage fixed

* Test Fix
 2022-05-26 15:15:41 +02:00
mosiac1
a47a2ce5b7
Add backend and UI support for direct joins between tables (#4943) 2022-05-25 23:41:26 -07:00