yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-11-25 23:22:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
7,167 Commits 922 Branches 182 Tags
Commit Graph

1695 Commits

Author SHA1 Message Date
Teddy
54fbe250a1
fix: import error + BQ E2E CLI (#12420) 2023-07-13 13:35:37 +02:00
Onkar Ravgan
e200a42926
Removed datamodels references from dashboard and datamodels (#12371) 2023-07-13 14:26:28 +05:30
Onkar Ravgan
90160ac761
Migrated DynamoDB to NoSQL type (#12404) 2023-07-13 14:25:31 +05:30
vanshika18
9174cea1ab
Add support for Superset datamodels (#12183) 2023-07-13 14:10:37 +05:30
Pere Miquel Brull
a3bff293a4
Fix #12190 - Bring support for Airflow 2.6 & Use Airflow 2.6.3 as the new ingestion base (#12398) 
* Bump Airflow version

* Support Airflow 2.6

* Lint

* Bump airflow version

* Fix airflow 2.6 lineage

* Fix airflow 2.6 lineage
 2023-07-13 06:41:59 +02:00
Teddy
b89cf64f14
Clean up profiler (#12369) 
* ref: implemented interface for profiler components + removed struct logic

* ref: ran python linting

* ref: added UML diagram to readme.md

* ref: empty commit for labeler check

* ref: remove multiple context manager for 3.7 3.8 compatibility

* ref: remove
 2023-07-12 17:02:32 +02:00
Pere Miquel Brull
1b7c5e3233
Fix #12321 - Deprecate impala scheme from Hive (#12327) 
* Deprecate impala scheme from Hive

* Add migrations
 2023-07-11 17:38:32 +02:00
Pere Miquel Brull
6a1b1190a7
Fix #12313 - Airflow lineage parsing and schedules (#12339) 
* Fix docs

* Fix lineage and intervals

* Format
 2023-07-11 07:35:22 +02:00
gauthk6
59c384f4e9
Fixes 11973: Pass required parameters while patching the description of a TestCase (#12244) 
* Fixes a bug while patching the description of a TestCase

* Update docker-compose.yml

* Update docker-compose.yml

* Ran pre-commit checks and linter

* Added some clarifying points and fixed some grammatical errors in the documentation for installation instructions.

* revert changes made to docs
 2023-07-07 09:51:23 +02:00
Ayush Shah
9ae3407be0
Fix Filter Pattern Delete issue (#12293) 2023-07-06 10:18:44 +05:30
Megumi AIKAWA
9aede5e821
Fixes 11924 : Add pgspider support (#12159) 
* Add PGSpider service connector

* add sqlalchemy-pgspider to setup.py

* Delete unnecessary code and move files.

* Change to work as a Postgres connector sub-module

* Rebuild PGSpiderLineage class into a set of funcs

---------

Co-authored-by: Pham Ngoc Son <son1.phamngoc@toshiba.co.jp>
 2023-07-05 09:18:59 +02:00
Hung Duong
64f147c517
Fixes #12129 : enhance bigquery sample data query (#12130) 
* upgrade BigQuery Sampler

* beautify code

* revert old way of profiler & data quality, keep fetch new way sample

* Update profiler_source.py

* Update profiler_source.py

---------

Co-authored-by: hung.duong <hung.duong@be.com.vn>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
 2023-07-04 08:37:15 +02:00
Mohit Yadav
466e325e0a
Added Server migration table to utility (#12263) 
* Added Server migration table to utility

* updated to be in not to migrate

* py checkstyle failure
 2023-07-03 19:45:50 +02:00
Onkar Ravgan
48b98d0be8
default gen time for comp time (#12256) 2023-07-03 12:30:02 +05:30
Pere Miquel Brull
b59ff24531
Fix looker view parser (#12254) 2023-07-03 07:25:08 +02:00
Teddy
93d132de5c
fix: removed SampleDataConnection class from package (#12229) 2023-07-01 11:21:21 -07:00
Onkar Ravgan
2a5a355c5f
fixed dbt table level test (#12247) 2023-07-01 19:00:41 +02:00
Pere Miquel Brull
1ecf5607c7
Looker - Fix file extension and blob import (#12232) 
* Fix file extension and blob import

* Fix file extension and blob import
 2023-06-29 16:14:17 +02:00
Onkar Ravgan
c1a379e202
made tableau name fields optional (#12223) 2023-06-29 14:04:07 +05:30
Onkar Ravgan
acf25f4555
Added pagination Tableau data sources graphql (#12187) 
* Added pagination tableau graphql

* changed downstream workbook
 2023-06-28 14:57:09 +02:00
Teddy
caa78a2310
fix: updated MySQL and Postres conn in backup/restore (#12191) 
* fix: updated MySQL and Postres conn in backup/restore

* fix: ran linting
 2023-06-28 13:46:29 +05:30
Ayush Shah
9e0a84e904
Fix Profiler Workflow infinite loop (#12185) 
* Fix Profiler Workflow infinite loop
 2023-06-27 20:19:02 +05:30
Ayush Shah
81a152140c
fix connection kwargs error in profiler (#12181) 2023-06-27 19:44:53 +05:30
Pere Miquel Brull
c22c5871f8
Update logging messages for JWT tokens (#12169) 2023-06-27 12:32:08 +02:00
Teddy
45a5ff49c8
fix: check on undefined bool value of clause (#12162) 2023-06-26 20:31:37 +02:00
Pere Miquel Brull
a9135e8db5
Mask queries, Table sample data and Topic sample data (#12139) 
* Filter out PII queries

* Mask topic PII

* Mask topic PII

* Update sample data test

* Format

* Moved logic to repository or PIIMasker

* Fix test

* chore(ui): update sample data api endpoint for topic

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
 2023-06-26 11:47:39 +02:00
Ayush Shah
29d3de2520
Move Top Level Imports (#12145) 2023-06-26 11:54:25 +05:30
Mayur Singal
a3fd6e9522
Fix #11659: Add support for filter patterns in dbt workflow (#12063) 2023-06-26 11:30:35 +05:30
Mayur Singal
18c8eb318e
Databricks Support Table Constraints (#12138) 
* Databricks Support Table Constraints

* pylint fix

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2023-06-26 07:01:53 +02:00
Mayur Singal
7d1b123efe
Fix atlas lineage custom db name issue (#12108) 
* Fix atlas lineage custom db name issue

* Fix test
 2023-06-25 19:16:00 -07:00
Onkar Ravgan
3a9d8a9a45
powerbi loop bug fix (#12120) 2023-06-25 18:20:59 +02:00
Pere Miquel Brull
97e08ee25c
Fix #12106 - Fix looker chart sourceUrl & Array datatype handling (#12113) 
* Fix looker chart url

* Handle array datatype

* Handle array datatype
 2023-06-25 08:18:36 -07:00
Aniket Katkar
67ffa1cce3
fix(UI): reverted the ES config step for 'Data Insights Pipeline' (#12121) 
* Reverted the ES config step for Data Insights pipeline

* es sink fixes for ingestion

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
 2023-06-24 18:15:51 +05:30
Onkar Ravgan
d9d3f6895b
Fix 6874: Added Support for lineage from dbt ephemeral nodes (#12101) 
* fixed dbt ephemeral nodes lin

* fixed dbt tests
 2023-06-23 10:01:22 +02:00
Mayur Singal
258a2e33e2
Fix #12078: Add clientSessionKeepAlive Field for snowflake (#12107) 2023-06-23 10:00:53 +02:00
Pere Miquel Brull
b5c4d5a9d3
Fix #10175 - Add Oracle lineage and usage (#12090) 
* Add Oracle lineage and usage

* Add docs

* Add examples
 2023-06-23 06:52:34 +02:00
Teddy
6f33465b7c
fix: move service retrival to workflow (#12109) 2023-06-23 06:40:32 +02:00
Ayush Shah
cb6e42941a
Fix 12025: Clickhouse NaN issue (#12079) 2023-06-22 12:51:56 +05:30
Onkar Ravgan
5197682921
Fixed dagster bugs and Added Pydantic Models (#12048) 2023-06-22 10:59:09 +05:30
Onkar Ravgan
78d7faf04a
Added Azure Blob Storage to dbt config (#11839) 2023-06-22 10:58:38 +05:30
Pere Miquel Brull
3b8e5e69fd
Fix #12056 - Clean verbose debug logs (#12060) 
* Clean verbose debug logs

* Format

* Remove unnecessary test

* remove verbosity
 2023-06-22 06:54:09 +02:00
Teddy
1e86b6533c
Fixes #11743 - Remove SQLParse dependency for System Metrics (#12072) 
* fix: removed sqlparse dependency for system metrics

* fix: update sample query

* fix: move system test os retrieval to `.get()`

* fix: move os.environ to `get`
 2023-06-22 06:51:24 +02:00
Onkar Ravgan
8fac9d8976
Added Powerbi Lineage across workspaces and datamodel projects (#12052) 
* powerbi lin multiple workspace

* Addressed review comments
 2023-06-21 17:16:09 +02:00
Onkar Ravgan
0cfd8d29c5
Added dbt test_metadata check (#12067) 2023-06-21 12:17:26 +02:00
Ayush Shah
9fa7b78d87
Issue 11686: Add IncludeOwner + scheduleInterval in Airflow (#12005) 2023-06-21 14:18:45 +05:30
Onkar Ravgan
47514f6c5f
Fixed dbt test col name (#12065) 2023-06-21 10:47:54 +02:00
Mayur Singal
c3cec54be9
Fix for #11807 Part 2: Add SourceUrl for table entity (#12013) 
* Fix #11807 Part 2: Add SourceUrl for table entity

* address review comments: centralize sourceurl

* remove qlick

* pytest fix

* fix typo
 2023-06-20 11:46:45 +02:00
Pere Miquel Brull
8751427679
Add project to Data Models JSON Schema & Implement in Looker (#12030) 
* Add project to dashboards and models

* Add project to dashboards and models

* Update openmetadata-spec/src/main/resources/json/schema/entity/data/dashboardDataModel.json

Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>

* Update openmetadata-spec/src/main/resources/json/schema/api/data/createDashboardDataModel.json

Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>

* Fix tests

---------

Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
 2023-06-20 11:45:29 +02:00
Mayur Singal
2738310887
Fix #12012: Fix clickhouse usage schema_name error (#12032) 2023-06-20 11:41:28 +05:30
Ayush Shah
83e9b6c310
Fixes 10395: Validation of yaml workflow configs (#11985) 2023-06-20 11:20:59 +05:30