yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-07-13 20:18:24 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,395 Commits 677 Branches 144 Tags
Commit Graph

1489 Commits

Author SHA1 Message Date
Pere Miquel Brull
f2fb0521c2
Update airflow loggers and rename ometa loggers (#9868) 
* Update airflow loggers and rename ometa loggers

* ANSI print to logger

* Remove colored logging from tests

* Merge ometa_logger into the one used in loggers class

* linting

* linting

Co-authored-by: Nahuel Verdugo Revigliono <nahuel@getcollate.io>
 2023-01-23 16:28:17 +01:00
Teddy
871cd0414e
Fix Partition for tables with profile sample + added mssql e2e tests (#9862) 
* fix: partition for sampled data

* fix: added MSSQL E2E cli test

* fix: added secerets varibale to action file

* fix: python checkstyle
 2023-01-23 14:16:44 +01:00
Mayur Singal
5145dced25
Datalake parquet files fix (#9860) 2023-01-23 17:44:34 +05:30
Nahuel
93057e476f
Fix: Superset encryption and decryption not working as expected (#9827) 
* Fix superset sample data

* Update ingestion/examples/superset_data/service.json

* Update ingestion/examples/superset_data/service.json

* fix: pytest error (#9824)

* fix: pytest error

* fix: linting

* increased verbosity

* empty commit to re-run tests

* print registry and test definition set

* renamed columnValuesToBeUnique fqn

* removed print statements + verbosity

* Docs Using a Custom Ingestion Image in helm (#9835)

* Docs Using a Custom Ingestion Image in helm

* Update openmetadata-docs/content/deployment/kubernetes/index.md

Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>

* Update openmetadata-docs/content/deployment/kubernetes/index.md

Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>

* Update openmetadata-docs/content/deployment/kubernetes/index.md

* Update index.md

Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>

* fix(ui): ui changes for #9695 (#9770)

* Fix #9695: Teams show up in lower case

* changed owner advanced search filter field from name to display name

* fixed minor quick filter bug

* fixed failing cypress tests

* fixed failing unit tests

* Fixed failing cypress tests

* added comments in unit tests for better understanding of changes

* fixed failing cypress tests for advanced search

* improved logic for showing advanced search options

Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>

* Added powerbi docs for new field (#9836)

* Fix#9460: Avoid reuse inspector to get view definition (#9821)

* Avoid reuse inspector to get view definition

* Update openmetadata-sqllineage version

* Feat: Update GH action for E2E tests (#9839)

* Update E2E CI tests GH action

* Address PR comments

* Add converter for services when encypting or decrypting services objects

* Update E2E CI tests GH action

Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
 2023-01-20 21:32:38 +00:00
Pere Miquel Brull
16a1b2c8be
Vertica comments, dbs, profiler and docs (#9845) 
* Vertica comments, dbs, profiler and docs

* Revert metabase changes

* Format

* Fix median
 2023-01-20 20:36:03 +01:00
Teddy
7d0eeeb802
Fix datetime fn mysql (#9825) 
* fix: mysql datetime function

* empty commit to re-run tests
 2023-01-20 14:33:55 +00:00
Nahuel
254ee9a186
Fix#9460: Avoid reuse inspector to get view definition (#9821) 
* Avoid reuse inspector to get view definition

* Update openmetadata-sqllineage version
 2023-01-20 13:54:41 +00:00
Teddy
dcf220f867
fix: pytest error (#9824) 
* fix: pytest error

* fix: linting

* increased verbosity

* empty commit to re-run tests

* print registry and test definition set

* renamed columnValuesToBeUnique fqn

* removed print statements + verbosity
 2023-01-20 10:45:11 +01:00
Mayur Singal
939adf887b
Fix #8973: Superset fetch metadata from DB (#9645) 
* Docs #8973: Superset with SSO docs

* Superset support for mysql & postgres db

* remove unwanted field from yaml

* Fix pylint

* Fix superset tests

* Fix sample data connection

* ci fix & review comments

* cypress & defualt provider fix
 2023-01-19 16:39:43 +01:00
Nahuel
ddff6e2875
Fix: Replace sqllineage with openmetadata-sqllineage (#9800) 
* Replace sqllineage with openmetadata-sqllineage

* Fix checkstyle and failing test

* Move logic to retrieve dialect of a service type into a class

* Improve py-check message when it fails

* Updated mapper

* Update code after merge
 2023-01-19 14:56:29 +01:00
Teddy
fd4d39ac05
fix: max value for column between (#9820) 2023-01-19 12:13:35 +00:00
Ayush Shah
14100d089f
Modify Charts (#9814) 2023-01-19 12:03:21 +00:00
Pere Miquel Brull
c6cf9ae764
Fix region name (#9811) 2023-01-19 12:49:18 +01:00
Nahuel
04dae8bc75
Revert "Fix #8884 & #9460 - View lineage happens in the lineage workflow (#9796)" (#9817) 
This reverts commit e88fe2f559989dbb1c34662abcdc35f235adc8c4.
 2023-01-19 11:56:21 +01:00
Pere Miquel Brull
b0058bdb65
Fix #9707 - Install instantclient and enable thick mode (#9761) 
* Install instantclient

* Prep instant client

* Update docs

* Amend comments
 2023-01-19 08:08:56 +01:00
Mayur Singal
f58765b7cc
Fix #6577: Optimize metadata reads (#9759) 2023-01-19 06:29:14 +00:00
Pere Miquel Brull
e88fe2f559
Fix #8884 & #9460 - View lineage happens in the lineage workflow (#9796) 
* View lineage happens in the lineage workflow

* Format

* Remove table views from context

* lint
 2023-01-18 18:32:40 +01:00
Ayush Shah
02147b6502
Fix: Quicksight Dashboard and chart Metadata Ingestion (#9786) 
* Fix Quicksight json schema and errors

* Fix Quicksight lineage

* add pagination

* Generalize Pagination

* Fix Pylints

* update max results

* fix tests

* Add Pydantic Models for QuickSight resp

* Add fqn builder

* Fix Py lInt

* Fix Tests
 2023-01-18 20:37:41 +05:30
NiharDoshi99
f2649041f2
Fix: atlas test connection (#9791) 
* Fix: atlas test connection

* Fix: docs changes

* Fix: docs changes

* Fix: python checkstyle

* Fix: python test
 2023-01-18 20:16:07 +05:30
Milan Bariya
7f111584b8
Deprecate OMetaDatabaseAndTable (#9760) 
* Deprecate OMetaDatabaseAndTable

* Change Based On Comments

* Change Based On Comments

* Fix Pytest

* Fix Pytest
 2023-01-18 14:29:32 +01:00
Onkar Ravgan
043f1e0601
Dbt e2e test (#9767) 
* Added base files

* dbt e2e tests

* Added tag condition based on UI feedback

* Added owner and lineage tests

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2023-01-18 14:29:16 +01:00
Abhishek Pandey
6d7acdc140
Issue 9542 - Replace the default name for MySQL db (#9687) 
* database-field-added-for-custom-database-name

* database-field-added-for-custom-database-name

* database field description-added

* databaseName-field-updated-in-connection-json

* database-name-field-updated-in-salesforce

* databaseName-description-updated

* database-name-logic-updated-in-common-db-source
 2023-01-18 14:28:40 +01:00
Onkar Ravgan
08a4d33c5f
Powerbi pagination fix (#9784) 
* Fixed powerbi pagination logic

* Added yaml changes
 2023-01-18 17:17:13 +05:30
Pere Miquel Brull
9c237d9565
Fix #9777 - Pass XLets as input parameters for lineage runner (#9780) 
* Pass XLets as input parameters for lineage runner

* Format
 2023-01-18 11:50:23 +01:00
Abhishek Pandey
86a57293ef
databricks-test-added (#9749) 
* databricks-test-added

* databricks-test-added
 2023-01-18 08:45:46 +01:00
Pere Miquel Brull
294277708b
Fix #9558 - Add a greater range for boto3 dependency (#9778) 
* add boto3 wiggle room

* add boto3 wiggle room
 2023-01-18 08:20:40 +01:00
Sriharsha Chintalapani
2a314809c1
Keep elasticsearch version to be 7.13.1 (#9756) 
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2023-01-17 19:12:49 -08:00
Suresh Srinivas
ac157a8f0b
Fixes #9618 Allow apostrophe in entity names (#9642) 
* Fixes #9618 Allow apostrophe in entity names

* Add support for space in entity names (with exception) and column names

* Add regex checks to entity names, table column names to allow only required characters

* fix: allowed characters in entityName

* Add all the allowed special characters in the entity name in tests

* Allow '&', '/', ':' only for the table

* Override basic entityName by defining entity specific entityName for Python code gen

* fix: dagster migration + json schema path

* fix: python style check

* Fix test failure

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
 2023-01-17 11:34:29 -08:00
NiharDoshi99
2954f17832
Fix: Domo dashboard issue (#9743) 
* Fix: Domo dashboard issue

* Fix: added execption handling

* Fix: changed as per comment
 2023-01-17 21:27:15 +05:30
Ayush Shah
f11cdbfab1
Profile sample rows + Randomization of chunks for datalake (#9595) 2023-01-16 22:17:46 +05:30
Teddy
45dde7a4bb
fix (profiler/dq): partitioning logic (#9731) 2023-01-14 22:35:51 +01:00
Mohit Yadav
b0ace9446d
remove computePercentile from table usage summary (#9681) 
* remove computePercentile from table usage summary

* Fix: removed the compute percentile and added at the end

Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: Nihar Doshi <nihardoshi16@gmail.com>
 2023-01-13 17:16:15 +01:00
NiharDoshi99
029dbe892e
Fix: added test case for atlas (#9678) 
* Fix: added test case for atlas

* Fix: resolved conflict

* Fix: changing back neo4j to old version

* Fix: changing back neo4j to old version

* Fix: changes as per comment

* Fix: changes as per comment

* Fix: python checkstyle
 2023-01-13 16:07:29 +05:30
Corey Zuares
c3197beaf4
Extra headers for metadata API (#9321) 
* Configurable Extra Authorization Header

- Adds two new header that can be set via env-var
  (`OMETA_HEADER_EXTRA_AUTH_NAME` and `OMETA_HEADER_EXTRA_AUTH_VALUE`)
- By default, it copies the Authorization header value to the extra auth header
- If a value is specified in the corresponding env-var, that value is
  used instead of the existing token

The primary use case is for Google Cloud's Identity Aware Proxy (IAP) which allows secure access to private services but removes the Authorization header in the process. The `Proxy-Authorization` header gets forwarded along to the service as the `Authorization` header for the internal service.

* Use openMetadataConnection schema instead of env vars

* Disable too-many-locals for _request function

* Fix dict merge
 2023-01-13 09:59:44 +01:00
Teddy
6fb47061f8
fix (profiler): column compilation (#9705) 2023-01-13 08:12:04 +01:00
Milan Bariya
bb41fec08f
Add: Function for Postgres Tags (#9683) 
* Add: Function for Postgres Tags

* Update: Query for generate fqn
 2023-01-12 09:46:08 +01:00
Mayur Singal
c4a7965ffc
Fix #7156: Add Sample Data for Kafka and Redpanda (#9472) 
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2023-01-12 09:43:25 +01:00
Mohit Yadav
f2e54fe7b9
Fix Topic index to map properly (#9688) 
* Fix Topic index to map properly

* fix python side
 2023-01-11 18:59:05 +00:00
Teddy
41ce2406a4
fixes #9673 - Oracle casting error (#9682) 
* fix (profiler): oracle profiler datatype error

* fix (profiler): linting

* fix: reorder connection type to default to db schema
 2023-01-11 19:46:10 +01:00
Teddy
903209bc70
Fixes #9656 -- Add support for datetime type for col between (#9671) 
* feat (profiler): Added support for datetime type for sqa profiler

* feat (profiler): migration file

* fix: added comment for ms timestamp logic
 2023-01-11 16:51:36 +01:00
Onkar Ravgan
4de2dacbe2
Fixed db2 profiler (#9679) 2023-01-11 20:40:25 +05:30
NiharDoshi99
1ec324e43e
Fix: neo4j version bump (#9680) 2023-01-11 18:28:25 +05:30
Pere Miquel Brull
e3ae3a8a33
[WIP] - Add Scarf gateway (#9652) 
* Update ingestion Readme with scarf

* Test scarf docker

* Update ingestion Readme with scarf

* Test scarf docker

* Add docker domain
 2023-01-11 09:33:09 +01:00
Pere Miquel Brull
bf753a4dee
Fix #7768 - Update and organize versions (#9664) 
Fix #7768 - Update and organize versions (#9664)
 2023-01-11 07:05:12 +01:00
Pere Miquel Brull
84348d4748
Fix #8866 - bump datamodel-codegen (#9623) 
* Fix #8866 - bump datamodel-codegen

* Update connection options and arguments structure

* Add builders test

* Format

* Allow Any values in componentConfig

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
 2023-01-09 13:20:32 +01:00
NiharDoshi99
27c68fb208
test case for DomoDatabase (#9630) 
* test case for domo-database

* test case for domo-database

* Fix: pytest
 2023-01-06 12:02:01 +01:00
Onkar Ravgan
e9ac8ecb85
Fix 7972: Added dbt tag classification name (#9575) 
* Added dbt tag classification name

* resolved merge conflicts

* handle ui side change for dbt schema change

* Fixed migration scripts

* fixed failing cypress for dbt

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
 2023-01-06 11:08:21 +05:30
Mayur Singal
5f4ae37238
Fix table comment not being ingested (#9627) 
* Fix table comment not being ingested

* lint fix

* Fixed DB2 comments

Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
 2023-01-05 18:44:49 +01:00
Mayur Singal
9ef32eef99
Fix #6829: Improve usage filtering (#9613) 2023-01-05 15:07:03 +00:00
Milan Bariya
809a228441
Fix: AttributeError when node is not in catalog (#9622) 
* Fix: AttributeError when node is not in catalog

* Change based on comments
 2023-01-05 18:57:54 +05:30