yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-08-06 08:08:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
10,673 Commits 700 Branches 151 Tags
Commit Graph

2916 Commits

Author SHA1 Message Date
Pere Miquel Brull
9c237d9565
Fix #9777 - Pass XLets as input parameters for lineage runner (#9780) 
* Pass XLets as input parameters for lineage runner

* Format
 2023-01-18 11:50:23 +01:00
Abhishek Pandey
86a57293ef
databricks-test-added (#9749) 
* databricks-test-added

* databricks-test-added
 2023-01-18 08:45:46 +01:00
Pere Miquel Brull
294277708b
Fix #9558 - Add a greater range for boto3 dependency (#9778) 
* add boto3 wiggle room

* add boto3 wiggle room
 2023-01-18 08:20:40 +01:00
Sriharsha Chintalapani
2a314809c1
Keep elasticsearch version to be 7.13.1 (#9756) 
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2023-01-17 19:12:49 -08:00
Suresh Srinivas
ac157a8f0b
Fixes #9618 Allow apostrophe in entity names (#9642) 
* Fixes #9618 Allow apostrophe in entity names

* Add support for space in entity names (with exception) and column names

* Add regex checks to entity names, table column names to allow only required characters

* fix: allowed characters in entityName

* Add all the allowed special characters in the entity name in tests

* Allow '&', '/', ':' only for the table

* Override basic entityName by defining entity specific entityName for Python code gen

* fix: dagster migration + json schema path

* fix: python style check

* Fix test failure

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
 2023-01-17 11:34:29 -08:00
NiharDoshi99
2954f17832
Fix: Domo dashboard issue (#9743) 
* Fix: Domo dashboard issue

* Fix: added execption handling

* Fix: changed as per comment
 2023-01-17 21:27:15 +05:30
Ayush Shah
f11cdbfab1
Profile sample rows + Randomization of chunks for datalake (#9595) 2023-01-16 22:17:46 +05:30
Teddy
45dde7a4bb
fix (profiler/dq): partitioning logic (#9731) 2023-01-14 22:35:51 +01:00
Mohit Yadav
b0ace9446d
remove computePercentile from table usage summary (#9681) 
* remove computePercentile from table usage summary

* Fix: removed the compute percentile and added at the end

Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: Nihar Doshi <nihardoshi16@gmail.com>
 2023-01-13 17:16:15 +01:00
NiharDoshi99
029dbe892e
Fix: added test case for atlas (#9678) 
* Fix: added test case for atlas

* Fix: resolved conflict

* Fix: changing back neo4j to old version

* Fix: changing back neo4j to old version

* Fix: changes as per comment

* Fix: changes as per comment

* Fix: python checkstyle
 2023-01-13 16:07:29 +05:30
Corey Zuares
c3197beaf4
Extra headers for metadata API (#9321) 
* Configurable Extra Authorization Header

- Adds two new header that can be set via env-var
  (`OMETA_HEADER_EXTRA_AUTH_NAME` and `OMETA_HEADER_EXTRA_AUTH_VALUE`)
- By default, it copies the Authorization header value to the extra auth header
- If a value is specified in the corresponding env-var, that value is
  used instead of the existing token

The primary use case is for Google Cloud's Identity Aware Proxy (IAP) which allows secure access to private services but removes the Authorization header in the process. The `Proxy-Authorization` header gets forwarded along to the service as the `Authorization` header for the internal service.

* Use openMetadataConnection schema instead of env vars

* Disable too-many-locals for _request function

* Fix dict merge
 2023-01-13 09:59:44 +01:00
Teddy
6fb47061f8
fix (profiler): column compilation (#9705) 2023-01-13 08:12:04 +01:00
Milan Bariya
bb41fec08f
Add: Function for Postgres Tags (#9683) 
* Add: Function for Postgres Tags

* Update: Query for generate fqn
 2023-01-12 09:46:08 +01:00
Mayur Singal
c4a7965ffc
Fix #7156: Add Sample Data for Kafka and Redpanda (#9472) 
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2023-01-12 09:43:25 +01:00
Mohit Yadav
f2e54fe7b9
Fix Topic index to map properly (#9688) 
* Fix Topic index to map properly

* fix python side
 2023-01-11 18:59:05 +00:00
Teddy
41ce2406a4
fixes #9673 - Oracle casting error (#9682) 
* fix (profiler): oracle profiler datatype error

* fix (profiler): linting

* fix: reorder connection type to default to db schema
 2023-01-11 19:46:10 +01:00
Teddy
903209bc70
Fixes #9656 -- Add support for datetime type for col between (#9671) 
* feat (profiler): Added support for datetime type for sqa profiler

* feat (profiler): migration file

* fix: added comment for ms timestamp logic
 2023-01-11 16:51:36 +01:00
Onkar Ravgan
4de2dacbe2
Fixed db2 profiler (#9679) 2023-01-11 20:40:25 +05:30
NiharDoshi99
1ec324e43e
Fix: neo4j version bump (#9680) 2023-01-11 18:28:25 +05:30
Pere Miquel Brull
e3ae3a8a33
[WIP] - Add Scarf gateway (#9652) 
* Update ingestion Readme with scarf

* Test scarf docker

* Update ingestion Readme with scarf

* Test scarf docker

* Add docker domain
 2023-01-11 09:33:09 +01:00
Pere Miquel Brull
bf753a4dee
Fix #7768 - Update and organize versions (#9664) 
Fix #7768 - Update and organize versions (#9664)
 2023-01-11 07:05:12 +01:00
Pere Miquel Brull
84348d4748
Fix #8866 - bump datamodel-codegen (#9623) 
* Fix #8866 - bump datamodel-codegen

* Update connection options and arguments structure

* Add builders test

* Format

* Allow Any values in componentConfig

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
 2023-01-09 13:20:32 +01:00
NiharDoshi99
27c68fb208
test case for DomoDatabase (#9630) 
* test case for domo-database

* test case for domo-database

* Fix: pytest
 2023-01-06 12:02:01 +01:00
Onkar Ravgan
e9ac8ecb85
Fix 7972: Added dbt tag classification name (#9575) 
* Added dbt tag classification name

* resolved merge conflicts

* handle ui side change for dbt schema change

* Fixed migration scripts

* fixed failing cypress for dbt

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
 2023-01-06 11:08:21 +05:30
Mayur Singal
5f4ae37238
Fix table comment not being ingested (#9627) 
* Fix table comment not being ingested

* lint fix

* Fixed DB2 comments

Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
 2023-01-05 18:44:49 +01:00
Mayur Singal
9ef32eef99
Fix #6829: Improve usage filtering (#9613) 2023-01-05 15:07:03 +00:00
Milan Bariya
809a228441
Fix: AttributeError when node is not in catalog (#9622) 
* Fix: AttributeError when node is not in catalog

* Change based on comments
 2023-01-05 18:57:54 +05:30
Steve Russo
3a3f248edd
Issue 5592: Add Databricks Pipeline Connector (#9554) 
* feat: update databricks_client

* feat: create pipelineconnection & connection

* test: add unit test

* feat: add examples file

* fix: remove unused imports

* style: fix python style

* fix: code review adjustments

* fix: remove unneeded schema ref

* fix: format requirements

* fix: change context calls

* test: add pipeline_status test

* format: pylint format change

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2023-01-05 14:14:12 +01:00
Ayush Shah
1d930ad14b
Fix security vulnerability (#9580) 2023-01-05 12:36:00 +05:30
Mayur Singal
0a34e18e26
Fix #9562: Add duration in usage ingestion (#9564) 2023-01-05 06:33:07 +00:00
NiharDoshi99
9145054dc6
Fix: refractor datalake datatypes and s3 for parquet (#9578) 
* Fix: refractor datalake datatypes

* Fix: s3 files for parquet
 2023-01-04 22:12:00 +05:30
Milan Bariya
625c15c5d3
Fix: Table description error for table (#9584) 
* Fix: Table description error for table

* add schema name in a query

* Add: Function for  Column comments

* Mssql qery import path changed

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2023-01-04 21:57:56 +05:30
Mayur Singal
67ea711319
Fix #8422: Fix False usage query count (#9587) 2023-01-04 14:01:07 +05:30
Pere Miquel Brull
1f49ea46b5
Fix #9149 - Postgres foreign tables & reorganize queries for sources (#9588) 
Fix #9149 - Postgres foreign tables & reorganize queries for sources (#9588)
 2023-01-04 08:50:53 +01:00
Mayur Singal
0a7fe62552
Redshift: varchar datatype fix (#9572) 2023-01-03 14:15:42 +05:30
8vicat
7e7528e13b
Improve SQL lineage parsing (#9537) 
* CTEs support
* Supports inline comments
* Expands `*` in sources
 2023-01-03 09:27:45 +01:00
NiharDoshi99
8f69386076
Fix: refactor get_column_type (#9559) 
* Fix: refactor get_column_type

* Fix: added changes as per comments

* Fix: pylint

* Fix: minor changes

* Fix: minor changes
 2023-01-03 10:28:38 +05:30
Pere Miquel Brull
7f21a7bced
Fix #8088 - Restructure source connections & clients (#9545) 2023-01-02 13:52:27 +01:00
Onkar Ravgan
420fa1181a
Fixed bigquery e2e test (#9555) 2023-01-02 13:08:47 +05:30
Onkar Ravgan
dc157ee286
Fixed ingestion tags with new tags API (#9552) 
* Fixed ingestion tags with new API

* fix cypress failures

* wait for tags API

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
 2022-12-31 07:33:02 -06:00
Ayush Shah
c08e805090
Revert Change (#9534) 
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2022-12-28 14:05:18 +01:00
Chirag Madlani
bf6fc5f93a
prepare(release) next release (#9479) 
* prepare(release) next release

* airflow typo

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2022-12-27 20:15:46 +05:30
Pere Miquel Brull
a4521fd664
Fix #6562 - Sources have their own package (#9521) 
Fix #6562 - Sources have their own package (#9521)
 2022-12-27 15:00:22 +01:00
Mayur Singal
e7133d1e86
Fix Elasticsearch reindex through CLI (#9528) 
* Fix Elasticsearch reindex through CLI

* Fix Elasticsearch reindex through CLI
 2022-12-27 12:06:05 +01:00
Suresh Srinivas
758c976cba
Fixes #9259 Change Tags APIs to conform with rest of the APIs (#9260) 2022-12-26 12:32:17 -08:00
Nahuel
2c43ebba6f
Fix#9448: Add ES volumes (#9506) 
* Add ES volumes

* Fix run_local_docker script

* Fix error run_local_docker script

* Update Es volumes in docker-compose files
 2022-12-23 17:33:30 +01:00
Pere Miquel Brull
ec6ebb3694
Organise dynamic importers (#9513) 
Organise dynamic importers (#9513)
 2022-12-23 12:17:06 +01:00
Mayur Singal
5be0634d86
Dagster YAML & Test connection Fix (#9489) 
* Dagster YAML Fix

* Test connection fix
 2022-12-22 11:00:27 +00:00
Sriharsha Chintalapani
b4e274c8d5
Add schemaText to topic Index (#9482) 
* Add schemaText to topic Index

* Remove ChangeDescription from TopicIndex
 2022-12-22 09:42:51 +01:00
Pere Miquel Brull
0e0779a926
Fix dict type hint for older versions (#9480) 2022-12-22 07:40:11 +00:00