yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-10-31 18:48:35 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,217 Commits 873 Branches 177 Tags
Commit Graph

888 Commits

Author SHA1 Message Date
Teddy
e026d625d6
Fixes #5713 where profileSample default to 100 (#5714) 
* Fix random sample modulo and rand for postgres

* Remove sampleProfile default and return declarative meta when no sample profile is passed

* fixed py_format

* isolated random sample partition logic into its own methid
 2022-06-28 19:27:55 +02:00
Pere Miquel Brull
aab7544778
Fix MSSQL modulo (#5707) 2022-06-28 15:53:16 +02:00
Pere Miquel Brull
89a026b022
Fix #5688 - MlFlow deployment fails from UI (#5691) 
* Fix mlflow from UI

* format
 2022-06-28 14:58:38 +02:00
Mayur Singal
4473280e63
Airbyte Status Improvments (#5701) 2022-06-28 14:32:37 +02:00
Mayur Singal
71bf32e438
Fix #5562: Handle Analysis Date for Usage (#5582) 
Fix #5562: Handle Analysis Date for Usage (#5582)
 2022-06-27 15:16:12 +02:00
Mayur Singal
369f8643d1
Fix #5619: Fix Multiple Source Column Level Lineage (#5620) 2022-06-27 16:38:46 +05:30
Milan Bariya
338d1efe40
FIX 5419: Glue pipeline connector not working (#5467) 
FIX 5419: Glue pipeline connector not working (#5467)
 2022-06-27 11:38:41 +02:00
Sriharsha Chintalapani
8528c51b9c
Fix #5516: Refactor elasticsearch indexes to be same as entity specifications (#5573) 
* Fix #5516: Refactor elasticsearch indexes to be same as entity specifications

Co-authored-by: darth-coder00 <aashit@getcollate.io>
Co-authored-by: Sachin-chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
 2022-06-27 01:19:27 -07:00
Mayur Singal
e640db3192
Fix #5525: Fixed Bigquery Comment Issue (#5616) 2022-06-26 12:17:33 -07:00
Teddy
9380b93a18
Fixed clickhouse bugs in profiler (#5624) 2022-06-26 12:17:01 -07:00
Pere Miquel Brull
b950f0fd06
Add mappings for table ES index ingestion (#5635) 2022-06-25 11:41:51 +02:00
Onkar Ravgan
932d1adfe5
added custom oidc to airflow provider (#5598) 
Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
 2022-06-24 20:42:19 -07:00
Milan Bariya
95bca61719
FIX Mssql ingestion Fail (#5627) 2022-06-24 20:41:35 -07:00
Sriharsha Chintalapani
710e782f33
Fix #5455: Ingestion from a kafka topic fails when sample data is enabled (#5536) 
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
 2022-06-24 20:39:41 -07:00
Sriharsha Chintalapani
ad0c6dad88
Fix #5613: Add fernet encryption and remove connection details for non admin users (#5614) 
* Fix #5613: Add fernet encryption and remove connection details for non-admin users
 2022-06-24 16:44:37 -07:00
Pere Miquel Brull
bb4ac68ee3
Add backend connection for airflow dag extraction 2022-06-24 20:01:50 +02:00
Teddy
7431c1c226
[Issue-5188] Implement User Query for Sampler and Profiler (#5578) 
* Added custom query sample for sample data ingestion

* Added logic to run table profiling against user's query

* Added tests for user query logic in profiler and sampler

* Added user profiling to tableProfile + fixed format

* staging commit

* Added logic to add profileQuery to table entity

* Added limit to sample rows
 2022-06-24 14:46:34 +02:00
Pere Miquel Brull
0144a1d46a
Fix #5423 & #3412 - Looker Lineage and Usage (#5581) 
Fix #5423 & #3412 - Looker Lineage and Usage (#5581)
 2022-06-23 13:01:40 +02:00
Pere Miquel Brull
7fa47c31d8
Fix #5565 - Handle missing column in usage fqn match (#5575) 
* add try

* Add some exception handling

* fix date and split
 2022-06-23 13:01:27 +02:00
Abhishek Pandey
a913cf46db
table-type-fixed-added (#5549) 
* table-type-fixed-added

* optimised-column-type
 2022-06-23 11:13:22 +05:30
Onkar Ravgan
2b25c299e5
Using context for charts (#5576) 
Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
 2022-06-22 10:45:28 +02:00
Pere Miquel Brull
685b7bede8
Revert "Added pipeline metadata to the edge lines (#5552)" (#5571) 
This reverts commit d717f8cc17d949620935a89d5fa94b141502ab07.
 2022-06-22 11:42:48 +05:30
Mayur Singal
841bb39d3b
Fix #5553: Clean cache support added in topology runner (#5554) 
* Fix #5553: Clean cache support added in topology runner

* used get_ctx_default
 2022-06-22 08:02:52 +02:00
Mayur Singal
9fbe07d2d0
Fix #5544: Downgrading Kafka Version (#5545) 2022-06-21 13:22:04 -07:00
Mario Taddeucci
a3f809d789
Fixing airflow lineage configs scopes (#5507) 2022-06-21 18:11:58 +02:00
Pere Miquel Brull
0ecc9f0da6
Fix #5459 - Remove sql-metadata in favor of sqllineage (#5494) 
Fix #5459 - Remove sql-metadata in favor of sqllineage (#5494)
 2022-06-21 18:02:50 +02:00
Mayur Singal
ba1b099301
[ISSUE-5198]: Clean stage file after processing (#5555) 2022-06-21 17:24:57 +02:00
Onkar Ravgan
d717f8cc17
Added pipeline metadata to the edge lines (#5552) 
Added pipeline metadata to the edge lines (#5552)
 2022-06-21 17:22:37 +02:00
Pere Miquel Brull
a14de6eee1
Fix #5367 - MlFlow connector & MLServices (#5446) 
Fix #5367 - MlFlow connector & MLServices (#5446)
 2022-06-21 14:54:36 +02:00
Teddy
5067e24374
[ISSUE-4723] Fix Snowflake Case Sensitive Error with Profiler (#5533) 
* Fixed snowflake profiler + enabled profiler integration tests

* Fixed py formating
 2022-06-20 22:23:17 +02:00
Onkar Ravgan
1c709ed0f9
Fixed mode test connection (#5531) 
Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
 2022-06-20 22:51:22 +05:30
Teddy
c151a34c70
[ISSUE-4659] Add Additional GE Tests to the Profiler (#5506) 
* Added table tests 'column count to be between' and 'column name to be in set'

* Added 'column values to be in set' test

* Fixed wrong filed name when invoking TestCaseResult

* Added col min and col max to be between

* Fix formatting

* Fix test after adding column name list to table profile

* Added 'column values sum between' test

* Added 'table column name to exist' test

* Fixed JSON description white space + spelling mistake
 2022-06-20 08:54:42 +02:00
Abhishek Pandey
9c8a58f345
dataclass-added-in-airbyte-client (#5512) 2022-06-20 06:33:45 +02:00
Onkar Ravgan
21dac15176
Added Mode Connector (#5475) 
* rebase with main

* Added lineage for mode

* added lineagerunner and enum constants

* added host port

* refactored mode according to new dashboard topology

* checkstyle fixes

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
 2022-06-19 23:59:38 +05:30
Abhishek Pandey
196b023643
airflow-test-connection-fixed (#5502) 
* airflow-test-connection-fixed

* airbyte-test-connection-added
 2022-06-17 23:08:18 +05:30
Mayur Singal
1d6b7e8525
Fix #3289: Added precision details of column (#5437) 
* Fix #3289: Added precision details of column

* Resolved review comments
 2022-06-17 14:12:54 +02:00
Mayur Singal
f53a07815e
Fix #5409: Implement Dashboard Topology (#5495) 
* Fix #5409: Dashboard Service

* Fix #5409: Implement Topology to Dashboards

* Removed print
 2022-06-17 15:22:54 +05:30
Mayur Singal
0ee87b2d99
Fix #5484: Postgress Connector Fix (#5485) 2022-06-16 16:11:50 +05:30
Mayur Singal
430d32a9c6
Fix #5090: Added Local TableType (#5450) 
Fix #5090: Added Local TableType (#5450)
 2022-06-15 10:41:48 +02:00
Abhishek Pandey
e8975aac01
datalake-csv-files-ingestion-added (#5343) 
datalake-csv-files-ingestion-added (#5343)
 2022-06-15 08:57:21 +02:00
Pere Miquel Brull
0a9e36f1b4
Fix Snowflake get comments query (#5451) 2022-06-14 18:02:30 -07:00
Teddy
28336b0a43
Fix #3575 Data Quality - Partitioned Tables (#5441) 
Co-authored-by: Teddy Crepineau <teddycrepineau@Teddys-MacBook-Pro.local>
Co-authored-by: Vivek Ratnavel Subramanian <vivekratnavel90@gmail.com>
 2022-06-14 12:37:44 -07:00
Onkar Ravgan
5537db2ab4
Added DBT Cloud support (#5436) 
Added DBT Cloud support (#5436)
 2022-06-13 18:05:13 +02:00
Milan Bariya
ab7689d7a8
FIX 5131: Fix Get Pipeline Status from Airflow metadata connector (#5396) 
FIX 5131: Fix Get Pipeline Status from Airflow metadata connector (#5396)
 2022-06-13 17:44:21 +02:00
Pere Miquel Brull
02fece727b
Fix #5307 - Define a better topology for Ingestion Sources (#5366) 
Fix #5307 - Define a better topology for Ingestion Sources (#5366)
 2022-06-13 12:17:54 +02:00
Mayur Singal
8ac83509a1
Fix #4549: Added Snowflake Session Query Tag (#5404) 
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
 2022-06-13 10:19:48 +02:00
Pere Miquel Brull
d9d30b2cba
Fix #5416 - Revisit Looker Test Connection (#5422) 
Fix #5416 - Revisit Looker Test Connection (#5422)
 2022-06-13 08:07:04 +02:00
Mayur Singal
51a275c481
Fix #1121: Added Airbyte Source (#5372) 
Fix #1121: Added Airbyte Source (#5372)
 2022-06-13 06:45:25 +02:00
Onkar Ravgan
7c387d6d54
Optimized Snowflake fetch tags (#5376) 
* optimized snowflake fetch tags

* fixed same named entities issue

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
 2022-06-09 22:24:40 +05:30
Snyk bot
c717cb1545
fix: ingestion/tests/integration/lineage/airflow/requirements.txt to reduce vulnerabilities (#5178) 
The following vulnerabilities are fixed by pinning transitive dependencies:
- https://snyk.io/vuln/SNYK-PYTHON-FLASKAPPBUILDER-2433100
 2022-06-08 13:36:31 -07:00