yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-09-15 20:13:14 +00:00
Code Issues Packages Projects Releases Wiki Activity
10,284 Commits 785 Branches 159 Tags
Commit Graph

2362 Commits

Author SHA1 Message Date
Teddy
871cd0414e
Fix Partition for tables with profile sample + added mssql e2e tests (#9862) 
* fix: partition for sampled data

* fix: added MSSQL E2E cli test

* fix: added secerets varibale to action file

* fix: python checkstyle
 2023-01-23 14:16:44 +01:00
Mayur Singal
5145dced25
Datalake parquet files fix (#9860) 2023-01-23 17:44:34 +05:30
Pere Miquel Brull
16a1b2c8be
Vertica comments, dbs, profiler and docs (#9845) 
* Vertica comments, dbs, profiler and docs

* Revert metabase changes

* Format

* Fix median
 2023-01-20 20:36:03 +01:00
Teddy
7d0eeeb802
Fix datetime fn mysql (#9825) 
* fix: mysql datetime function

* empty commit to re-run tests
 2023-01-20 14:33:55 +00:00
Nahuel
254ee9a186
Fix#9460: Avoid reuse inspector to get view definition (#9821) 
* Avoid reuse inspector to get view definition

* Update openmetadata-sqllineage version
 2023-01-20 13:54:41 +00:00
Mayur Singal
939adf887b
Fix #8973: Superset fetch metadata from DB (#9645) 
* Docs #8973: Superset with SSO docs

* Superset support for mysql & postgres db

* remove unwanted field from yaml

* Fix pylint

* Fix superset tests

* Fix sample data connection

* ci fix & review comments

* cypress & defualt provider fix
 2023-01-19 16:39:43 +01:00
Nahuel
ddff6e2875
Fix: Replace sqllineage with openmetadata-sqllineage (#9800) 
* Replace sqllineage with openmetadata-sqllineage

* Fix checkstyle and failing test

* Move logic to retrieve dialect of a service type into a class

* Improve py-check message when it fails

* Updated mapper

* Update code after merge
 2023-01-19 14:56:29 +01:00
Teddy
fd4d39ac05
fix: max value for column between (#9820) 2023-01-19 12:13:35 +00:00
Ayush Shah
14100d089f
Modify Charts (#9814) 2023-01-19 12:03:21 +00:00
Pere Miquel Brull
c6cf9ae764
Fix region name (#9811) 2023-01-19 12:49:18 +01:00
Nahuel
04dae8bc75
Revert "Fix #8884 & #9460 - View lineage happens in the lineage workflow (#9796)" (#9817) 
This reverts commit e88fe2f559989dbb1c34662abcdc35f235adc8c4.
 2023-01-19 11:56:21 +01:00
Pere Miquel Brull
b0058bdb65
Fix #9707 - Install instantclient and enable thick mode (#9761) 
* Install instantclient

* Prep instant client

* Update docs

* Amend comments
 2023-01-19 08:08:56 +01:00
Mayur Singal
f58765b7cc
Fix #6577: Optimize metadata reads (#9759) 2023-01-19 06:29:14 +00:00
Pere Miquel Brull
e88fe2f559
Fix #8884 & #9460 - View lineage happens in the lineage workflow (#9796) 
* View lineage happens in the lineage workflow

* Format

* Remove table views from context

* lint
 2023-01-18 18:32:40 +01:00
Ayush Shah
02147b6502
Fix: Quicksight Dashboard and chart Metadata Ingestion (#9786) 
* Fix Quicksight json schema and errors

* Fix Quicksight lineage

* add pagination

* Generalize Pagination

* Fix Pylints

* update max results

* fix tests

* Add Pydantic Models for QuickSight resp

* Add fqn builder

* Fix Py lInt

* Fix Tests
 2023-01-18 20:37:41 +05:30
NiharDoshi99
f2649041f2
Fix: atlas test connection (#9791) 
* Fix: atlas test connection

* Fix: docs changes

* Fix: docs changes

* Fix: python checkstyle

* Fix: python test
 2023-01-18 20:16:07 +05:30
Milan Bariya
7f111584b8
Deprecate OMetaDatabaseAndTable (#9760) 
* Deprecate OMetaDatabaseAndTable

* Change Based On Comments

* Change Based On Comments

* Fix Pytest

* Fix Pytest
 2023-01-18 14:29:32 +01:00
Onkar Ravgan
043f1e0601
Dbt e2e test (#9767) 
* Added base files

* dbt e2e tests

* Added tag condition based on UI feedback

* Added owner and lineage tests

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2023-01-18 14:29:16 +01:00
Abhishek Pandey
6d7acdc140
Issue 9542 - Replace the default name for MySQL db (#9687) 
* database-field-added-for-custom-database-name

* database-field-added-for-custom-database-name

* database field description-added

* databaseName-field-updated-in-connection-json

* database-name-field-updated-in-salesforce

* databaseName-description-updated

* database-name-logic-updated-in-common-db-source
 2023-01-18 14:28:40 +01:00
Onkar Ravgan
08a4d33c5f
Powerbi pagination fix (#9784) 
* Fixed powerbi pagination logic

* Added yaml changes
 2023-01-18 17:17:13 +05:30
Pere Miquel Brull
9c237d9565
Fix #9777 - Pass XLets as input parameters for lineage runner (#9780) 
* Pass XLets as input parameters for lineage runner

* Format
 2023-01-18 11:50:23 +01:00
Suresh Srinivas
ac157a8f0b
Fixes #9618 Allow apostrophe in entity names (#9642) 
* Fixes #9618 Allow apostrophe in entity names

* Add support for space in entity names (with exception) and column names

* Add regex checks to entity names, table column names to allow only required characters

* fix: allowed characters in entityName

* Add all the allowed special characters in the entity name in tests

* Allow '&', '/', ':' only for the table

* Override basic entityName by defining entity specific entityName for Python code gen

* fix: dagster migration + json schema path

* fix: python style check

* Fix test failure

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
 2023-01-17 11:34:29 -08:00
NiharDoshi99
2954f17832
Fix: Domo dashboard issue (#9743) 
* Fix: Domo dashboard issue

* Fix: added execption handling

* Fix: changed as per comment
 2023-01-17 21:27:15 +05:30
Ayush Shah
f11cdbfab1
Profile sample rows + Randomization of chunks for datalake (#9595) 2023-01-16 22:17:46 +05:30
Teddy
45dde7a4bb
fix (profiler/dq): partitioning logic (#9731) 2023-01-14 22:35:51 +01:00
Mohit Yadav
b0ace9446d
remove computePercentile from table usage summary (#9681) 
* remove computePercentile from table usage summary

* Fix: removed the compute percentile and added at the end

Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: Nihar Doshi <nihardoshi16@gmail.com>
 2023-01-13 17:16:15 +01:00
NiharDoshi99
029dbe892e
Fix: added test case for atlas (#9678) 
* Fix: added test case for atlas

* Fix: resolved conflict

* Fix: changing back neo4j to old version

* Fix: changing back neo4j to old version

* Fix: changes as per comment

* Fix: changes as per comment

* Fix: python checkstyle
 2023-01-13 16:07:29 +05:30
Corey Zuares
c3197beaf4
Extra headers for metadata API (#9321) 
* Configurable Extra Authorization Header

- Adds two new header that can be set via env-var
  (`OMETA_HEADER_EXTRA_AUTH_NAME` and `OMETA_HEADER_EXTRA_AUTH_VALUE`)
- By default, it copies the Authorization header value to the extra auth header
- If a value is specified in the corresponding env-var, that value is
  used instead of the existing token

The primary use case is for Google Cloud's Identity Aware Proxy (IAP) which allows secure access to private services but removes the Authorization header in the process. The `Proxy-Authorization` header gets forwarded along to the service as the `Authorization` header for the internal service.

* Use openMetadataConnection schema instead of env vars

* Disable too-many-locals for _request function

* Fix dict merge
 2023-01-13 09:59:44 +01:00
Teddy
6fb47061f8
fix (profiler): column compilation (#9705) 2023-01-13 08:12:04 +01:00
Milan Bariya
bb41fec08f
Add: Function for Postgres Tags (#9683) 
* Add: Function for Postgres Tags

* Update: Query for generate fqn
 2023-01-12 09:46:08 +01:00
Mayur Singal
c4a7965ffc
Fix #7156: Add Sample Data for Kafka and Redpanda (#9472) 
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2023-01-12 09:43:25 +01:00
Mohit Yadav
f2e54fe7b9
Fix Topic index to map properly (#9688) 
* Fix Topic index to map properly

* fix python side
 2023-01-11 18:59:05 +00:00
Teddy
41ce2406a4
fixes #9673 - Oracle casting error (#9682) 
* fix (profiler): oracle profiler datatype error

* fix (profiler): linting

* fix: reorder connection type to default to db schema
 2023-01-11 19:46:10 +01:00
Teddy
903209bc70
Fixes #9656 -- Add support for datetime type for col between (#9671) 
* feat (profiler): Added support for datetime type for sqa profiler

* feat (profiler): migration file

* fix: added comment for ms timestamp logic
 2023-01-11 16:51:36 +01:00
Onkar Ravgan
4de2dacbe2
Fixed db2 profiler (#9679) 2023-01-11 20:40:25 +05:30
NiharDoshi99
1ec324e43e
Fix: neo4j version bump (#9680) 2023-01-11 18:28:25 +05:30
Pere Miquel Brull
bf753a4dee
Fix #7768 - Update and organize versions (#9664) 
Fix #7768 - Update and organize versions (#9664)
 2023-01-11 07:05:12 +01:00
Pere Miquel Brull
84348d4748
Fix #8866 - bump datamodel-codegen (#9623) 
* Fix #8866 - bump datamodel-codegen

* Update connection options and arguments structure

* Add builders test

* Format

* Allow Any values in componentConfig

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
 2023-01-09 13:20:32 +01:00
Onkar Ravgan
e9ac8ecb85
Fix 7972: Added dbt tag classification name (#9575) 
* Added dbt tag classification name

* resolved merge conflicts

* handle ui side change for dbt schema change

* Fixed migration scripts

* fixed failing cypress for dbt

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
 2023-01-06 11:08:21 +05:30
Mayur Singal
5f4ae37238
Fix table comment not being ingested (#9627) 
* Fix table comment not being ingested

* lint fix

* Fixed DB2 comments

Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
 2023-01-05 18:44:49 +01:00
Mayur Singal
9ef32eef99
Fix #6829: Improve usage filtering (#9613) 2023-01-05 15:07:03 +00:00
Milan Bariya
809a228441
Fix: AttributeError when node is not in catalog (#9622) 
* Fix: AttributeError when node is not in catalog

* Change based on comments
 2023-01-05 18:57:54 +05:30
Steve Russo
3a3f248edd
Issue 5592: Add Databricks Pipeline Connector (#9554) 
* feat: update databricks_client

* feat: create pipelineconnection & connection

* test: add unit test

* feat: add examples file

* fix: remove unused imports

* style: fix python style

* fix: code review adjustments

* fix: remove unneeded schema ref

* fix: format requirements

* fix: change context calls

* test: add pipeline_status test

* format: pylint format change

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2023-01-05 14:14:12 +01:00
Mayur Singal
0a34e18e26
Fix #9562: Add duration in usage ingestion (#9564) 2023-01-05 06:33:07 +00:00
NiharDoshi99
9145054dc6
Fix: refractor datalake datatypes and s3 for parquet (#9578) 
* Fix: refractor datalake datatypes

* Fix: s3 files for parquet
 2023-01-04 22:12:00 +05:30
Milan Bariya
625c15c5d3
Fix: Table description error for table (#9584) 
* Fix: Table description error for table

* add schema name in a query

* Add: Function for  Column comments

* Mssql qery import path changed

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2023-01-04 21:57:56 +05:30
Mayur Singal
67ea711319
Fix #8422: Fix False usage query count (#9587) 2023-01-04 14:01:07 +05:30
Pere Miquel Brull
1f49ea46b5
Fix #9149 - Postgres foreign tables & reorganize queries for sources (#9588) 
Fix #9149 - Postgres foreign tables & reorganize queries for sources (#9588)
 2023-01-04 08:50:53 +01:00
Mayur Singal
0a7fe62552
Redshift: varchar datatype fix (#9572) 2023-01-03 14:15:42 +05:30
8vicat
7e7528e13b
Improve SQL lineage parsing (#9537) 
* CTEs support
* Supports inline comments
* Expands `*` in sources
 2023-01-03 09:27:45 +01:00