yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-08-07 00:28:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,395 Commits 700 Branches 151 Tags
Commit Graph

1489 Commits

Author SHA1 Message Date
Sriharsha Chintalapani
ceeefb40be
Fix #2524: Ingestion: DBT model parsing fails with column (#2525) (#2639) 2022-02-06 12:49:57 -08:00
Teddy
b5ddada930
Fix #1526: Added OMetaTag mixin 2022-02-06 12:42:13 -08:00
Ayush Shah
21491559de
Fix #2587: Bigquery ADC and Policy Tag implementation 2022-02-06 10:36:10 -08:00
Mayur Singal
e1d7d644b3
Fix #323: fixes in table_usage.py (#2576) 
Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
 2022-02-06 10:34:56 -08:00
Pere Miquel Brull
c2ad7f6373
Fix #2317: Override Pipeline tasks from client 2022-02-06 10:17:08 -08:00
Vivek Ratnavel Subramanian
e6343a79d7
Fix #2555: PUT APIs should only allow admin and owners to update a resource in secure env (#2608) 2022-02-04 12:39:08 -08:00
Mayur Singal
6128fa30dc
Fix #2620: pk_columns & unique_columns fixed (#2621) 
Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
 2022-02-04 12:38:12 -08:00
Mayur Singal
5a6e7d743e
ISSUE-545: Added tests for QueryProcessor (#2510) 
Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
 2022-02-04 08:20:20 -08:00
Mayur Singal
bc13e343a1
issue-2572: added comments for vertica columns (#2596) 
* issue-2572: added comments for vertica columns

* moved sql queries to sql_queries.py

* fixed sonar bug and sql query

Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
 2022-02-04 15:43:27 +05:30
SeungwanJo
9a0ffc08e4
add ingestion hive col_comment (#2594) 2022-02-04 15:43:14 +05:30
Ayush Shah
513a113187
Sample Teams ingestion fixed (#2554) 
* Sample Teams ingestion fixed

* Error Handling in creating teams and fixed secretstr

* modified sample_data for teams
 2022-02-03 13:58:59 +05:30
Pere Miquel Brull
9887e0773e
Fix #2516 - User Profile & Amundsen Source (#2546) 2022-02-01 13:55:26 -08:00
Ayush Shah
763f125861
Bigquery Os unlink issue resolved (#2544) 
* Bigquery Os unlink issue resolved

* Bigquery code modified
 2022-02-01 23:33:08 +05:30
Robert Pack
67a1c23482
Fix #2348: Harmonize request titles in json schemas 2022-01-31 16:29:56 -08:00
Ayush Shah
73d848ac3b
Docker Typeerror fixed (#2504) 2022-01-31 12:30:12 +05:30
Pere Miquel Brull
6ea6f0c43d
Fix #2383 - Multiple validation workflows (#2500) 
* Clean duplicated validation

* Move black dep to dev

* Fix black version for ingestion core

* Validate core generated in the same workflow
 2022-01-30 12:26:41 -08:00
Pere Miquel Brull
d10639958c
Fix black version 21.12b0 (#2499) 2022-01-30 06:54:11 -08:00
Sriharsha Chintalapani
f6ac393007
Issue #2492: Improve messaging on docker command (#2493) 
* Issue #2492: Improve messaging on docker command

* Issue #2492: Improve messaging on docker command
 2022-01-29 09:47:21 +05:30
Vivek Ratnavel Subramanian
52ec2e3ad1
Fix #2478: Update the auth header to Authorization Bearer for JWT token (#2483) 
* Fix #2478: Update the auth header to Authorization Bearer for JWT token
 2022-01-27 23:58:54 -08:00
Teddy
4f3e330dd8
Issue 997-01 (#2476) 
* Fix linting

* Fixed pyformat errors

* Address comments from PR review

* Added back phony in makefile

* Added back comment
 2022-01-27 18:45:45 -08:00
Ayush Shah
3a6217a7e9
Fixed data profiling and raw data type issue (#2474) 
* Fixed data profiling and raw data type issue

* Update metadata_rest.py

* Update sql_source.py
 2022-01-27 18:59:35 +05:30
Mayur Singal
a8bdf844dd
make black (#2475) 
Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
 2022-01-27 18:58:30 +05:30
Sriharsha Chintalapani
c36ae72ccd
Issue #2462: Fix profiler group_by query (#2463) 2022-01-27 14:26:32 +05:30
Sriharsha Chintalapani
4cd5c2bea2
Issue #2457: remove logging utils (#2458) 2022-01-26 10:26:17 -08:00
Ayush Shah
02f3678f2a
Fixed Mysql profiler + removed additional dependency for profiler (#2447) 
* Fixed Mysql profiler + removed additional dependency for profiler

* Update sql_source.py

* Update sql_source.py

* Update setup.py
 2022-01-26 16:45:28 +05:30
Akash Jain
20309cde64
fix:cherry pick from 0.8.0 and miscellaneous fixes (#2445) 
* updating ingestion-core version to 0.8.0

* updating ingestion version to 0.8.0

* fixing file copy path

* update docker images to point to 0.8.0

* set up new release version

* reset ingestion version to 0.9.0.dev0

* reset ingestion-core version to 0.9.0
 2022-01-26 14:00:27 +05:30
Mayur Singal
cfca795edf
ISSUE-2422: Handled None datatype (#2423) 
Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
 2022-01-25 08:42:37 -08:00
Sriharsha Chintalapani
0e736012a9
Fix #1843: Add Delta Lake Connector (#1844) 
* Fix #1843: Add Delta Lake Connector

* Fix #1843: Add Delta Lake Connector
 2022-01-25 14:35:44 +05:30
Ayush Shah
917ccd7147
Bigquery and Helpers fix - Ingestion (#2408) 
* Bigquery Ingestion fix - error handling and keyerror

* Update helpers.py and column_type_parser

* Update helpers.py
 2022-01-25 14:30:09 +05:30
Pere Miquel Brull
ce5df0eb88
Use path str properly (#2405) 2022-01-25 08:55:44 +01:00
Ayush Shah
b4d63bd951
Fix: Typo in setting up pyarrow dependency (#2398) 2022-01-25 11:58:10 +05:30
Ayush Shah
b689c2d632
Multiple fixes - Looker, Ingestion fix, pyarrow dependency (#2385) 
* Added pyarrow dependency

* Fix #2378

* Fix Looker

* Fix Looker, secretstr to str
 2022-01-24 12:29:54 -08:00
Pere Miquel Brull
05c8d1ebb6
Fix #2153 - Lineage date parsing (#2387) 
* Check for start_date

* Fix date keys and remove repr

* Add date management tests
 2022-01-24 12:29:16 -08:00
Pere Miquel Brull
6a8fc9e0fb
import on call (#2382) 2022-01-24 12:29:03 -08:00
Pere Miquel Brull
b39b698eb7
Fix #2345 - Make bump & unify (#2359) 
* Add installing instructions

* Unify makefile and automate bump version setup

* Update submake call

* Mark targets as phony

* Add help to recipes

* Update core publish

* Do not trigger install from echo

* Run make from root

* Prepare py_format from isort and black

* Run py_format

* Check black and isort formatting

* Update comment msg
 2022-01-23 09:47:06 -08:00
Ayush Shah
6f5a4eafe7
Bigquery support for env without passing any config (#2160) 
* Bigquery support for env without passing any config

* Bigquery File formatted
 2022-01-22 22:25:48 -08:00
Sriharsha Chintalapani
e5fdf77d8e
Fix #2355: Upgrade scripts, handle entity_extension updatedAt and dbservice jdbc field , Fix #2353: Add API to AirflowPipelines filter based on service (#2357) 
* Fix #2340: Ingestion pipeline deployment refactor - update ingestion

* Fix #2355: Upgrade scripts, handle entity_extension updatedAt and dbservice jdbc field , Fix #2353: Add API to AirflowPipelines filter based on service

* Fix #2355: Upgrade scripts, handle entity_extension updatedAt and dbservice jdbc field , Fix #2353: Add API to AirflowPipelines filter based on service
 2022-01-22 21:12:25 -08:00
Pere Miquel Brull
1eb3f1ad41
[issue-2322] - Backup CLI (#2349) 
* Prepare cli module

* Check call

* Prepare backup

* Run pytest on schema changes

* Remove traces

* Add backup plugin

* run generate during CI

* Install after generate

* Address logger hotspot
 2022-01-22 12:08:14 -08:00
ulixius9
8fca53ec52
ISSUE-2048: Bigquery data profiler uses schema name instead of project id (#2335) 
* ISSUE-2048: Bigquery data profiler uses schema name instead of project id

* Update bigquery.py

* Update bigquery.py

* Update profiler_runner.py

* Update profiler.py

Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
 2022-01-22 23:33:15 +05:30
Sriharsha Chintalapani
f390bac375
Fix 2270: Column Type Parser (#2271) 
* Fix 2270: Column Type Parser

* Fix 2270: Column Type Parser

* Added checks to allow arrayDataType and dataTypeDisplay

* Modified - arrayDataType and dataTypeDisplay

* Update sql_source.py

* Update sql_source.py

* file formatted

* Modified according to column_type_parser.py

* modified / refactored / deleted the files

* Added Tests, modified sql_source

* file formatted

* Added missing datatypes

* Added Tests

* Added Tests

* Added Tests - refactored expected output into a json file

* file formatted

* Sample Data Updated

Co-authored-by: Ayush Shah <ayush@getcollate.io>
 2022-01-22 23:32:58 +05:30
Pere Miquel Brull
6693b4ae40
[issue-2262] - Simplify sample data DAG (#2337) 
* Simplify sample data DAG

* Remove mkdir

* Generate sources before running compose

* Generate sources to install models to ingestion image

* Add python-on-whales for docker --start

* Remove python-on-whales from base

* Install venv

* Setup python
 2022-01-22 16:37:56 +05:30
Sriharsha Chintalapani
10e71a7e09
Fix #2340: Ingestion pipeline deployment refactor (#2341) 
* Fix #2340: Ingestion pipeline deployment refactor
 2022-01-21 22:06:14 -08:00
codingwithabhi
ddf4ae261b
Bigquery credential configuration updated (#2041) 
* bigquery-credentials-in-bigquery-json

* ui-changes-reverted

* credentials-moved-to-options

* Update bigquery.json

* Update bigquery.py

* delete-temp-code-added

* delete-temp-code-added

* removed-init-from-config

* Update datatypes_test.py

* Update column_helpers.py

* code-formatted

Co-authored-by: Abhishek <abhishek@Abhisheks-MacBook-Air.local>
Co-authored-by: = <=>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
 2022-01-21 17:23:02 +05:30
Matt
2a80caa8fe
Add logic to initialize relationships from seed data during application startup (#2307) 
* Add logic to initialize relationships from seed data during application startup

* Remove ingestion related code for access control policies

* Move PolicyEvaluator init to PolicyResource
 2022-01-20 22:18:03 -08:00
Sriharsha Chintalapani
18f5c288fe Fix #2320: Fix DB migration scripts to upgrade to 0.8 from 0.7 2022-01-20 15:08:03 -08:00
Sriharsha Chintalapani
20e6c4bc86 fix upgrade 2022-01-20 15:08:03 -08:00
ulixius9
d594499fa0
ISSUE-1838: Added table type in glue connector (#2309) 
* ISSUE-1838: Added table type in glue connector

* Initialized table type with regular

Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
 2022-01-20 07:57:44 -08:00
ulixius9
d6522e2ab4
ISSUE-2272: Ingestion: Roles and Teams apis called only when ingesting users (#2284) 
* ISSUE-2272: Ingestion: Roles and Teams apis called only when ingesting users

* reformatted metadata_rest.py

Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
 2022-01-20 12:24:35 +05:30
Sriharsha Chintalapani
0a3ba3ddfd
Fix 2276: fix elastic search indexing (#2278) 2022-01-18 23:30:36 -08:00
Sriharsha Chintalapani
0d3ded0742
Data Profiler Integration (#2235) 
* Fix 2234: Data profiler integration
 2022-01-18 20:25:43 -08:00