90 Commits

Author SHA1 Message Date
Pere Miquel Brull
43be01b92e
Fix #4091 & #3948 - Create services correctly (#4101) 2022-04-13 18:04:52 +02:00
Ayush Shah
1f3667b46f
Fix Python test (#4095) 2022-04-13 21:03:33 +05:30
Sriharsha Chintalapani
be836e5404
Fix #4071: PUT IngestionPipeline missing property & error message (#4085)
Fix #4071: PUT IngestionPipeline missing property & error message (#4085)
2022-04-13 08:40:21 +02:00
Pere Miquel Brull
6a6507e754
Fix #3962 - Profiler uses DatabaseSchema & Sample Data fix (#4056) 2022-04-12 13:40:59 +05:30
Pere Miquel Brull
b3480693e4
Fix #3824 - OMeta Schema and JSON Connections (#3861)
Fix #3824 - OMeta Schema and JSON Connections (#3861)
2022-04-05 21:20:39 +02:00
Pere Miquel Brull
e2539c5e83
Fix #3844 - First iteration for deprecating MetadataServerConfig (#3853)
* Style

* deprecate MetadataServerConfig

* Remove audience from Okta
2022-04-05 18:02:45 +05:30
Sriharsha Chintalapani
b14c8dc2c4
Issue-3685: Variable based separator used for fullyQualifiedName instead of hardcoded . for Python and make the separator to : (#3778)
* Issue-3685: Variable based separator used for fullyQualifiedName instead of hardcoded . for Python and make the sepearator to :

* Fix failing test

* Use colon for run_local_docker validation

* Update tests FQDN

* Update tests FQDN

Co-authored-by: Sachin-chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: pmbrull <peremiquelbrull@gmail.com>
2022-03-31 19:20:27 +05:30
Pere Miquel Brull
9ced748c4f
Use root for FQDN (#3780) 2022-03-31 12:05:11 +05:30
Pere Miquel Brull
548a0ab722
Fix #3437 & #3186 - Airflow lineage Task Group & Tests (#3460) 2022-03-21 10:29:49 -07:00
Ayush Shah
538ee87acc
Fix tests - added recursion delete and error handling (#3560) 2022-03-21 10:00:39 -07:00
Pere Miquel Brull
16e82d45de
Fix #3371 - Run Profiler and Tests on a % of the data (#3424)
Fix #3371 - Run Profiler and Tests on a % of the data (#3424)
2022-03-16 06:05:59 +01:00
Pere Miquel Brull
130bbb0c5c
Fix #3104 - Remove unused imports with pycln (#3370)
Fix #3104 - Remove unused imports with pycln (#3370)
2022-03-14 06:59:15 +01:00
Pere Miquel Brull
94d7500216
Fix #3248 & #3251 - Update metrics and column profile (#3262)
Fix #3248 & #3251 - Update metrics and column profile (#3262)
2022-03-08 11:44:39 +01:00
Pere Miquel Brull
2444b884bf
Fix #3150 - Missing count & value length tests (#3193)
Fix #3150 - Missing count & value length tests (#3193)
2022-03-07 07:19:13 +01:00
Francisco J. Jurado Moreno
a6e98b3fdb
Fix2113 - Database Service API Integration tests (#3185) 2022-03-06 14:59:32 +01:00
Pere Miquel Brull
71207de362
Fix #2875 - Profiler API Sink (#3011)
Fix #2875 - Profiler API Sink
2022-03-02 16:46:28 +01:00
Alberto Miorin
fe5618c8f1
Fix #3037: metadata --version doesn't work (#3038) 2022-03-01 12:19:36 +01:00
Pere Miquel Brull
4d928bab80
Fix #2965 - Additional Properties set to False (#2976) 2022-02-24 08:38:56 -08:00
Pere Miquel Brull
a4b383fa83
Fix #2897 - Profiler CLI (#2967)
Fix #2897 - Profiler CLI
2022-02-24 08:03:50 +01:00
Pere Miquel Brull
1b2ea54d4f
Fix #2316 - Append Tasks & Add Status in Airflow (#2738)
* Add license

* Add date to timestamp helper

* Prepare pipeline status operation

* Update and clean tasks from client

* Update tasks from client

* Check if pipeline is empty

* Keep all current pipeline info

* Append and clean tasks

* format

* Add status information

* Test pipelineStatus update

* Update task on clear

* Log status on callback

* Update lineage and status docs

* Update lineage docs

* Format

* Logic to handle DAG status

* Lint and format

* Update lineage tests
2022-02-13 17:51:25 +01:00
Vivek Ratnavel Subramanian
e6343a79d7
Fix #2555: PUT APIs should only allow admin and owners to update a resource in secure env (#2608) 2022-02-04 12:39:08 -08:00
Robert Pack
67a1c23482
Fix #2348: Harmonize request titles in json schemas 2022-01-31 16:29:56 -08:00
Sriharsha Chintalapani
10e71a7e09
Fix #2340: Ingestion pipeline deployment refactor (#2341)
* Fix #2340: Ingestion pipeline deployment refactor
2022-01-21 22:06:14 -08:00
Sriharsha Chintalapani
0d3ded0742
Data Profiler Integration (#2235)
* Fix 2234: Data profiler integration
2022-01-18 20:25:43 -08:00
Pere Miquel Brull
a9c840778a
[issue-2116] - Add target property to MlModel (#2134)
* Add target property to MlModel

* Use pull_request_target for forks

* Simplify str

* Update ingestion-core setup

* Bump ingestion-core version

* Update install
2022-01-18 16:27:35 +01:00
Pere Miquel Brull
178315d68a
[issue-2112] - Add helper to get EntityReference in OMeta API (#2114)
* Use entity list from mixin

* Add entity reference helper

* Add tests for retrieving the entity reference

* Add missing space

* Fix shadowing

* Use get entity ref

* Use get entity ref
2022-01-10 09:07:39 +01:00
Teddy
dbcc5e9dd2
ISSUE-1048: Add Version Mixin (#2018)
* Added entity version mixins logics to ometa API

* remove logging in line 385 used for testing

* Fixed black error + ran isort

* remove extra underscore in

* Added integration tests for OpenMetadata versions methods

* Fixed linting errors in versionMixin.py
2022-01-07 10:37:56 +01:00
Pere Miquel Brull
48ebcffbd0
[issue-1976] - Ingestion SonarCloud (#2085)
* Update path

* Prepare sonar properties

* Prepare coverage recipes

* Add coverage

* Simplify pytest

* Organise integration tests

* Update path

* Use setup instead of reqs

* Update recipes

* Fix PR event to target

* Update event_name

* Prepare sonar

* Run tests & sonar

* Use sonarcloud host

* Fix compose

* Use ingestion token
2022-01-07 10:28:38 +01:00
Pere Miquel Brull
eb01c55676
Update naming (#2080) 2022-01-06 14:42:50 +01:00
Pere Miquel Brull
d303be847e
Prepare MlModel lineage (#1879) 2021-12-21 17:44:29 -08:00
Pere Miquel Brull
185ef5e98f
Get id as str or uuid (#1562) 2021-12-06 08:40:53 +01:00
parthp2107
a9e70e6a89
License header update (#1498)
* updated liscense header

* updated license header

* updated license header

* updated license header

* addressing stylecheck findings
2021-12-01 12:46:28 +05:30
Pere Miquel Brull
ea79b47c45
[Issue-1268] - MlModel generated sources (#1288)
* Update ML model naming for generated python src

* Update MlModel id and title

* Update MlModel descriptions

* Generate pydantic models

* Update OpenMetadata with new MlModel naming

* Update MlModel naming

* Update MlModel naming

* Update MlModel namings for backend sources
2021-11-20 08:26:45 -08:00
Pere Miquel Brull
ffd7818978
[Issue-1099] - Update MLModel Ingestion (#1179)
* Refactor MLModel

* Prepare MLModel properties example

* Update API tags to MLModels

* Update API descriptions for MLModel

* Rename tags to mlModels
2021-11-15 07:58:15 -08:00
Pere Miquel Brull
87fd93a3d0
Add GET with fields param (#1033) 2021-11-02 13:04:55 -07:00
Pere Miquel Brull
11f9ddae3c
Issue 954 (#1020)
* Prepare mixins

* Remove tasks

* Prepare table mixin

* Clean tasks from sample_data

* Add table mixin tests

* Prepare compute percentile

* Refactor OpenMetadata
2021-11-01 08:57:53 -07:00
Pere Miquel Brull
423e81ee68
Generate python models (#990)
* Mark internal functions

* Regenerate python models

* Prepare health check
2021-10-31 08:23:01 -07:00
Pere Miquel Brull
42db57e3b8
Issue 953 (#972)
* Rename docstring

* Fix typo

* Prepare linage API

* Centralise GET API

* Fix delete test by FQDN

* Add hints

* Add lineage tests

* Migrate lineage ops to OpenMetadata API
2021-10-28 11:31:38 -07:00
Pere Miquel Brull
710675d51a
OpenMetadata High Level API (#942)
* update arg name for fqdn

* fix suite setup and teardown

* Update list logic

* Add Chart tests

* Add Dashboard tests

* Add pipeline tests

* Add table tests

* Simplify create API

* Formatting tests

* Update create API

* Add Tasks and Topics tests

* Rename OpenMetadata

* Add client close

* Use Optional for GET id and name

* Fix #931 fetching dashboards using client

* Add tests

* Fix sample users ingestion

* Remove lineage test

* Rename services for independence
2021-10-26 14:18:43 -07:00
Pere Miquel Brull
f91bcc03f6
[Issue-877] - High Level API (#890)
* First approach

* Test model

* Validate endpoints

* Update schema codegen

* Prepare reqs dev

* Rename titles and codegen

* Update README

* Keep lineage title as addLineage

* Update pydantic classes

* Update API wrapper & tests

* Handle services methods

* Prepare tests
2021-10-21 14:51:38 -07:00