yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-10-31 18:48:35 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,829 Commits 873 Branches 177 Tags
Commit Graph

79 Commits

Author SHA1 Message Date
Pere Miquel Brull
04421901c0
[WIP] Fix #4610 - Pipeline Tasks removal (#4615) 
Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
 2022-05-07 21:10:35 -07:00
Pere Miquel Brull
a22ff4627f
Fix #4728 - Fix Kafka test connection (#4730) 
* Fix Kafka connection

* Update defaults and required values

* Update test connection

* Add filter tests

* Fix kafka tests
 2022-05-06 08:59:05 +02:00
Ayush Shah
4d2bde4fe7
Fixes Tableau - added env with default value (#4408) 2022-04-23 21:53:13 +05:30
Ayush Shah
a94bb5fc8e
Fixes Tableau + fix Charts not getting ingested as part of dashboard (#4327) 
* Fixes Tableau + fix Charts not getting ingested as part of dashboard

* pytest fix

* Fix Tableau json config
 2022-04-22 12:49:55 +02:00
Mayur Singal
db0e34c709
Fixing Test Connection for Dynamo & Glue (#4316) 
* Fixing Test Connection for Dynamo

* Fixed Glue Connector

* renamed engine to connection

* Fixed the return signature

* Added dataclass
 2022-04-22 11:30:59 +05:30
Pere Miquel Brull
256b16d877
Fix #4032 - Bigquery properties & GCS Credentials (#4202) 
Fix #4032 - Bigquery properties & GCS Credentials (#4202)
 2022-04-19 12:31:34 +02:00
Milan Bariya
924e211ba1
ISSUE-4076: Use the central get_engine method for usage (#4180) 
* ISSUE-4076: Use the central get_engine method for usage

* ISSUE-4076: Use the central get_engine method for usage

* ISSUE-4076: Use the central get_engine method for usage

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
 2022-04-19 12:35:01 +05:30
Pere Miquel Brull
63415952e3
Fix sample data (#4200) 
Fix sample data (#4200)
 2022-04-18 19:00:36 +02:00
Pere Miquel Brull
2aecebd595
Fix #4189 - Fix airflow lineage callback (#4191) 
Fix #4189 - Fix airflow lineage callback (#4191)
 2022-04-18 15:13:26 +02:00
Pere Miquel Brull
3f4129d239
Prepare custom encoder for SecretStr (#4186) 2022-04-18 14:43:22 +05:30
codingwithabhi
8115c2ece4
issue-4135: updated-mysql-connection-json (#4136) 2022-04-14 14:16:09 -07:00
Pere Miquel Brull
6768bf708d
Fix Py tests (#4122) 
Fix Py tests (#4122)
 2022-04-14 11:22:39 +02:00
Pere Miquel Brull
43be01b92e
Fix #4091 & #3948 - Create services correctly (#4101) 2022-04-13 18:04:52 +02:00
Ayush Shah
1f3667b46f
Fix Python test (#4095) 2022-04-13 21:03:33 +05:30
Sriharsha Chintalapani
be836e5404
Fix #4071: PUT IngestionPipeline missing property & error message (#4085) 
Fix #4071: PUT IngestionPipeline missing property & error message (#4085)
 2022-04-13 08:40:21 +02:00
Pere Miquel Brull
6a6507e754
Fix #3962 - Profiler uses DatabaseSchema & Sample Data fix (#4056) 2022-04-12 13:40:59 +05:30
Pere Miquel Brull
06a3e4c989
Fix #3825 - Schema Name, SQL Source FQDN & ORM Profiler (#3942) 
* Fix db schema name

* Fix sqlite connection

* Correctly register scanned tables

* improve sqlite connection

* Adapt schemas on ORM profiler

* Format
 2022-04-08 19:28:10 +05:30
Pere Miquel Brull
93525aea0d
Fix #3823 & Fix generate default Enum field (#3909) 
* Fix sample data DAG

* Fix callback imports

* Use --set-default-enum-member in generate

* Format

* Add faulty merge hard_delete

* Fix airflow lineage, improve naming and fix lineage tests

* Add mysql url test

* Add mysql url test

* Update CI name

* Fix test ometa endpoint

* Format

* Fix metadata config
 2022-04-07 18:22:50 +05:30
Pere Miquel Brull
b3480693e4
Fix #3824 - OMeta Schema and JSON Connections (#3861) 
Fix #3824 - OMeta Schema and JSON Connections (#3861)
 2022-04-05 21:20:39 +02:00
Pere Miquel Brull
e2539c5e83
Fix #3844 - First iteration for deprecating MetadataServerConfig (#3853) 
* Style

* deprecate MetadataServerConfig

* Remove audience from Okta
 2022-04-05 18:02:45 +05:30
Sriharsha Chintalapani
b14c8dc2c4
Issue-3685: Variable based separator used for fullyQualifiedName instead of hardcoded . for Python and make the separator to : (#3778) 
* Issue-3685: Variable based separator used for fullyQualifiedName instead of hardcoded . for Python and make the sepearator to :

* Fix failing test

* Use colon for run_local_docker validation

* Update tests FQDN

* Update tests FQDN

Co-authored-by: Sachin-chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: pmbrull <peremiquelbrull@gmail.com>
 2022-03-31 19:20:27 +05:30
Pere Miquel Brull
9ced748c4f
Use root for FQDN (#3780) 2022-03-31 12:05:11 +05:30
Pere Miquel Brull
b3087d08b9
Fix #3522 - Add timeout to profiler (#3707) 
Fix #3522 - Add timeout to profiler (#3707)
 2022-03-30 08:54:27 +02:00
Snyk bot
3173646484
fix: ingestion/tests/integration/source/mlflow/Dockerfile to reduce vulnerabilities (#3618) 2022-03-23 18:26:41 -07:00
Pere Miquel Brull
548a0ab722
Fix #3437 & #3186 - Airflow lineage Task Group & Tests (#3460) 2022-03-21 10:29:49 -07:00
Ayush Shah
538ee87acc
Fix tests - added recursion delete and error handling (#3560) 2022-03-21 10:00:39 -07:00
Pere Miquel Brull
16e82d45de
Fix #3371 - Run Profiler and Tests on a % of the data (#3424) 
Fix #3371 - Run Profiler and Tests on a % of the data (#3424)
 2022-03-16 06:05:59 +01:00
Pere Miquel Brull
130bbb0c5c
Fix #3104 - Remove unused imports with pycln (#3370) 
Fix #3104 - Remove unused imports with pycln (#3370)
 2022-03-14 06:59:15 +01:00
Pere Miquel Brull
94d7500216
Fix #3248 & #3251 - Update metrics and column profile (#3262) 
Fix #3248 & #3251 - Update metrics and column profile (#3262)
 2022-03-08 11:44:39 +01:00
Pere Miquel Brull
2444b884bf
Fix #3150 - Missing count & value length tests (#3193) 
Fix #3150 - Missing count & value length tests (#3193)
 2022-03-07 07:19:13 +01:00
Pere Miquel Brull
4a752e3ab2
Fix #3151 - Ingestion profiler should use ORM Profiler (#3192) 2022-03-06 15:43:43 -08:00
Francisco J. Jurado Moreno
a6e98b3fdb
Fix2113 - Database Service API Integration tests (#3185) 2022-03-06 14:59:32 +01:00
Snyk bot
834bb8e483
fix: ingestion/tests/integration/source/mlflow/Dockerfile to reduce vulnerabilities (#3134) 
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-DEBIAN11-AOM-1290331
- https://snyk.io/vuln/SNYK-DEBIAN11-AOM-1298721
- https://snyk.io/vuln/SNYK-DEBIAN11-AOM-1300249
- https://snyk.io/vuln/SNYK-DEBIAN11-UNZIP-2396444
- https://snyk.io/vuln/SNYK-DEBIAN11-UNZIP-2396445
 2022-03-04 08:34:24 +01:00
Snyk bot
8132b91c30
fix: ingestion/tests/integration/lineage/airflow/requirements.txt to reduce vulnerabilities (#3130) 
The following vulnerabilities are fixed by pinning transitive dependencies:
- https://snyk.io/vuln/SNYK-PYTHON-APACHEAIRFLOW-2413442
- https://snyk.io/vuln/SNYK-PYTHON-APACHEAIRFLOW-2414089
- https://snyk.io/vuln/SNYK-PYTHON-WTFORMS-40581
 2022-03-04 07:49:58 +01:00
Pere Miquel Brull
71207de362
Fix #2875 - Profiler API Sink (#3011) 
Fix #2875 - Profiler API Sink
 2022-03-02 16:46:28 +01:00
Alberto Miorin
fe5618c8f1
Fix #3037: metadata --version doesn't work (#3038) 2022-03-01 12:19:36 +01:00
Pere Miquel Brull
990608522a
Fix #2981 - Update Profile to match TableProfile (#2982) 2022-02-25 09:26:30 -08:00
Pere Miquel Brull
4d928bab80
Fix #2965 - Additional Properties set to False (#2976) 2022-02-24 08:38:56 -08:00
Pere Miquel Brull
a4b383fa83
Fix #2897 - Profiler CLI (#2967) 
Fix #2897 - Profiler CLI
 2022-02-24 08:03:50 +01:00
Pere Miquel Brull
1224d20a36
Fix #2894 - Profiler Processor & Metrics (#2900) 
Fix #2894 - Profiler Processor & Metrics (#2900)
 2022-02-22 08:09:02 +01:00
Pere Miquel Brull
e55579aaa8
Fix #2845 - Init Profiler Workflow (#2862) 
* Fix list typing

* Add sqlite service

* Add sqlite service

* Add sqlite service

* Refactor validation into class

* Refactor validation into class

* Prepare table simple profiler

* Add note

* test ORM conversion

* Prepare workflow config utilities

* Prepare workflow skeleton

* Use new core Validation

* Refactor workflow config parsing

* Add comment

* Simplify workflow validations

* Fix table metric check

* Add init for convenience, otherwise interpreter cries when trying to __call__ the get result

* Fix table metric check

* Format

* Format

* Fix table list and metrics init

* Prepare profiler workflow integration tests

* Bump version

* Fix pycharm imports

* format
 2022-02-20 17:55:12 +01:00
Pere Miquel Brull
76f4ccd590
Fix #2735 - Simplify Airflow properties extraction (#2749) 
* Simplify lineage properties extraction

* Add network name

* Format
 2022-02-14 16:53:42 +01:00
Pere Miquel Brull
1b2ea54d4f
Fix #2316 - Append Tasks & Add Status in Airflow (#2738) 
* Add license

* Add date to timestamp helper

* Prepare pipeline status operation

* Update and clean tasks from client

* Update tasks from client

* Check if pipeline is empty

* Keep all current pipeline info

* Append and clean tasks

* format

* Add status information

* Test pipelineStatus update

* Update task on clear

* Log status on callback

* Update lineage and status docs

* Update lineage docs

* Format

* Logic to handle DAG status

* Lint and format

* Update lineage tests
 2022-02-13 17:51:25 +01:00
Vivek Ratnavel Subramanian
e6343a79d7
Fix #2555: PUT APIs should only allow admin and owners to update a resource in secure env (#2608) 2022-02-04 12:39:08 -08:00
Robert Pack
67a1c23482
Fix #2348: Harmonize request titles in json schemas 2022-01-31 16:29:56 -08:00
Pere Miquel Brull
05c8d1ebb6
Fix #2153 - Lineage date parsing (#2387) 
* Check for start_date

* Fix date keys and remove repr

* Add date management tests
 2022-01-24 12:29:16 -08:00
Sriharsha Chintalapani
10e71a7e09
Fix #2340: Ingestion pipeline deployment refactor (#2341) 
* Fix #2340: Ingestion pipeline deployment refactor
 2022-01-21 22:06:14 -08:00
Sriharsha Chintalapani
0d3ded0742
Data Profiler Integration (#2235) 
* Fix 2234: Data profiler integration
 2022-01-18 20:25:43 -08:00
Pere Miquel Brull
a9c840778a
[issue-2116] - Add target property to MlModel (#2134) 
* Add target property to MlModel

* Use pull_request_target for forks

* Simplify str

* Update ingestion-core setup

* Bump ingestion-core version

* Update install
 2022-01-18 16:27:35 +01:00
Pere Miquel Brull
2728041d0f
[issue-2213] - Airflow lineage refactor (#2218) 
* Fix ometa logging

* Refactor airflow lineage

* Refactor airflow lineage

* Add airflow lineage integration tests

* Fix circular dependency
 2022-01-14 13:00:33 -08:00