yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-09-08 00:18:36 +00:00
Code Issues Packages Projects Releases Wiki Activity
10,447 Commits 781 Branches 159 Tags
Commit Graph

2872 Commits

Author SHA1 Message Date
harshsoni2024
4ac6111893 salesforce table description from label if not through query (#18286) 2024-10-16 12:58:06 +05:30
harshsoni2024
7bbc92b960 MINOR: Fix pinotdb col. datatype error (#18268) 2024-10-16 12:57:58 +05:30
Mayur Singal
2ea683a112 MINOR: Fix Couchbase columns not fetched (#18284) 2024-10-16 09:55:17 +05:30
Ayush Shah
36f682753c Fixes #18186: Quicksight Ingestion Error handled (#18218) 2024-10-16 09:55:13 +05:30
Imri Paran
5f93b05d8b MINOR: add column case sensitivity parameter (#18115) 
* fix(data-quality): table diff

- added handling for case-insensitive columns
- added handling for different numeric types (int/float/Decimal)
- added handling of boolean test case parameters

* add migrations for table diff

* add migrations for table diff

* removed cross type diff for now. it appears to be flaky

* fixed migrations

* use casefold() instead of lower()

* - implemented utils.get_test_case_param_value
- fixed params for case sensitive column

* handle bool test case parameters

* format

* testing

* format

* list -> List

* list -> List

* - change caseSensitiveColumns default to fase
- added migration to stay backward compatible

* - removed migration files
- updated logging message for table diff migration

* changed bool test case parameters default to always be false

* format

* docs: data diff

- added the caseSensitiveColumns parameter

requires: https://github.com/open-metadata/OpenMetadata/pull/18115

* fixed test_get_bool_test_case_param

(cherry picked from commit be82086e2542d2d176ac66e0bf11100646448b4f)
 2024-10-15 16:41:43 +02:00
Imri Paran
58b11669aa cherry picked be82086e2542d2d176ac66e0bf11100646448b4f 2024-10-15 16:40:02 +02:00
harshsoni2024
83bfda5229 GEN-1552: Postgres stored procedures support (#18083) 2024-10-15 14:26:29 +05:30
harshsoni2024
a9a37cfd0b fix query for mysql con. (#18272) 2024-10-15 14:21:04 +05:30
Mayur Singal
84300d8b56 Fix #17963: Fix PinotDB Ingestion (#18266) 
* Fix #17963: Fix PinotDB Ingestion

* fix conn args
 2024-10-15 08:40:57 +05:30
sushi30
bf9a098aa6 Merge remote-tracking branch 'origin/1.5.7' into 1.5.7 2024-10-14 11:39:48 +02:00
Imri Paran
c68cfb0a20 MINOR: fix(looker): exclude version 
https://github.com/looker-open-source/sdk-codegen/issues/1518
(cherry picked from commit 3c7f995677214fbc25c3c6c331f903105b48a0f9)
 2024-10-14 11:39:10 +02:00
Suman Maharana
fe2531e448 GEN-895: Added Glue Pipeline Lineage (#18063) 2024-10-14 14:55:19 +05:30
Suman Maharana
63bff7aafa Fixed dbt Manifest and Run results parsing (#18234) 2024-10-14 14:55:09 +05:30
IceS2
8c5d9ab0ca
MINOR: Update DB2 dependencies to fix issue about not having ibm-db installed (#18192) 
* Update DB2 dependencies to fix issue about not having ibm-db installed

* Fix checkstyle
 2024-10-10 08:25:59 +02:00
Prajwal214
e5d43bce96 Minor: Updated pyiceberg version to 0.5.1 (#18155) 
Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>
 2024-10-08 18:09:20 +05:30
Akash Jain
c51ae56d9b
fix: PostgreSQL installation in Ingestion Docker (#18114) 2024-10-08 12:34:25 +02:00
IceS2
8640b4c665
MINOR: Fix Ingestion Dockerfile Compatibility by changing IF syntax (#18098) 
* Fix Ingestion Dockerfile Compatibility by changing IF syntax

* Increased shell compatibility by using only one '='
 2024-10-07 15:08:25 +02:00
harshsoni2024
075c67b8b3 get primitive datatype in case of array, struct (#18102) 2024-10-07 11:12:06 +05:30
github-actions
6a36355d3e chore(release): Prepare Branch for 1.5.7 2024-10-03 15:01:32 +00:00
Onkar Ravgan
b1dcb1180e AlationSink conn improvements (#18091) 2024-10-03 16:23:09 +05:30
Mayur Singal
84441c4ad1 GEN-1251: Fix Expected ColumnLineage but got dict (#17708) 2024-10-03 14:30:50 +05:30
Aldwyn Cabarrubias
312b584836 Change time format for the created_at of the DBT cloud pipeline status (#17916) 
Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
 2024-09-30 17:02:38 +05:30
Teddy
d3325cda93 MINOR - Fix General Profiler Bugs (#17995) 
* fix import issue

* fix: better handle None values in profiler processing

* fix: profiler errors

* chore: fix comment

* style: fix python linting failure

* fix: null byte error with the database client

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
(cherry picked from commit 8dc6b7d282a700c2a3d4659f904a3966e16843a2)
 2024-09-30 08:13:08 +02:00
IceS2
cb2ef3f559 Fix missing cast to str (#18016) 2024-09-30 07:55:49 +05:30
Suman Maharana
2348614695 Added DBT tests with versionless and fixed v7 parsing (#18028) 2024-09-27 19:57:13 +05:30
Suman Maharana
25bda0f756 Minor: fix lineage & queries in dbt (#17999) 2024-09-27 19:48:56 +05:30
Suman Maharana
5f234cdbc4 Fix Manifest is not parsed correctly on dbt versionless (#17975) 2024-09-27 19:47:46 +05:30
harshsoni2024
27816cb964 allow powerbi datamodel children in col.lineage (#18030) 2024-09-27 19:30:08 +05:30
Mayur Singal
3e5cc923bf MINOR: MSTR connector import fix (#18025) 2024-09-27 16:57:35 +05:30
github-actions
95bfb5b912 chore(release): Prepare Branch for 1.5.6 2024-09-25 08:11:38 +00:00
Pere Miquel Brull
fe2681f022 FIX #17896 - Python lineage SDK to work with Uuid & FQN models (#17928) 2024-09-23 11:21:31 +05:30
Pere Miquel Brull
c75378f8e8 Merge branch '1.5.5' of github.com:open-metadata/OpenMetadata into 1.5.5 2024-09-20 16:07:16 +02:00
Pere Miquel Brull
ad03f9e237 GEN-996 - Allow PII Processor without storing Sample Data (#17927) 
* GEN-996 - Allow PII Processor without storing Sample Data

* fix import

* fix import
 2024-09-20 16:07:11 +02:00
Ayush Shah
c90b7b0be5 Fixes #17869: Add validations in place for salesforce connection (#17870) 2024-09-20 18:00:07 +05:30
Imri Paran
6b3f4646c6 ci: pin spacy version (#17814) 
spacy 3.8 requires numpy 2.0 which is not compatible with openmetadata-ingestion requirements:
184e508d9c

(cherry picked from commit 3282b057d84630dc04f9967678446db420d95bac)
 2024-09-20 12:19:27 +02:00
Pere Miquel Brull
01e4b04573 MINOR - Bump Presidio Analyzer and validate support for legal entities (#17750) 2024-09-20 10:36:44 +02:00
Imri Paran
e22a66fe45 Fix 17903: fix(data-quality): snowflake data diff (#17907) 
* fix(data-quality): snowflake data diff

- fixed schema in snowflake URL for data diff
- added e2e for snowflake data quality

* reverted unintended change

(cherry picked from commit 21af02d8d5d65ce73645b5396b3cd2eaee1fac19)
 2024-09-20 08:17:29 +02:00
Mayur Singal
eda9ecc83f MINOR: Apply multi threading in View Lineage Processing (#17922) 2024-09-19 18:57:31 +05:30
Pere Miquel Brull
9f78b09c79 GEN-1493 - Fix paginate_es in opensearch (#17858) 
* GEN-1493 - Fix opensearch pagination

* GEN-1494 - Add CI for py-tests with Postgres and Opensearch

* GEN-1494 - Add CI for py-tests with Postgres and Opensearch
 2024-09-19 10:27:51 +02:00
Pere Miquel Brull
4e7b1381b1 GEN-1410 - Improve PII Logging information (#17835) 2024-09-19 10:27:20 +02:00
github-actions
c55b8efe36 chore(release): Prepare Branch for 1.5.5 2024-09-16 03:19:15 +00:00
Imri Paran
6cd5a43fc4 [GEN-1406]: import plugin and clear plugin message for failed source import (#17788) 
* support side effects on source classes by always importing source class

* streamlined error message

* fixed service type extraction for test suite pipeline

* - replaced "custom" with constant
- added quotes for the plugin exception for copy/paste ergonomics

(cherry picked from commit 59854de58d000aae01bcc29c7194f7388614bdda)
 2024-09-13 11:12:58 +02:00
Mayur Singal
05cb36f8db MINOR: make type optional in ES Response (#17832) 
* MINOR: make type optional in ES Mixin

* make none as default

* pyformat
 2024-09-13 10:25:35 +02:00
Pere Miquel Brull
69fcd6a572 MINOR - Add ES pagination with error handling (#17776) 
* MINOR - Add ES pagination with error handling

* format

* format

* add nullable

* prepare API

* fix pagination

* format
 2024-09-12 09:42:23 +02:00
Mayur Singal
e23a18b88c MINOR: Fix Redshift View Def regex_replace Error (#17810) 2024-09-11 21:32:10 +05:30
github-actions
53cc84aef1 chore(release): Prepare Branch for 1.5.4 2024-09-10 05:59:03 +00:00
Pere Miquel Brull
f9dd7b53cb MINOR - Fix lineage GET for names with / and standardize quote calls (#17748) 
* MINOR - Fix lineage GET for names with `/` and standardize quote calls

* format

* fix import
 2024-09-06 14:53:17 +02:00
Pere Miquel Brull
fa198f2942 MINOR - Better PII classification for JSON data (#17734) 
* MINOR - Better PII classification for JSON data

* linting
 2024-09-06 14:52:14 +02:00
Onkar Ravgan
f829c506c1 Fixed dbt cloud run_res name (#17694) 2024-09-05 10:40:36 +05:30
Pere Miquel Brull
75be6b69dd MINOR - PII Scanner tests and log levels (#17686) 
* MINOR - PII Scanner tests and log levels

* MINOR - PII Scanner tests and log levels
 2024-09-04 14:27:30 +02:00