yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-09-03 14:13:06 +00:00
Code Issues Packages Projects Releases Wiki Activity
13,006 Commits 771 Branches 157 Tags
Commit Graph

3385 Commits

Author SHA1 Message Date
Ayush Shah
ad7258e7be
Fixes 10949: return Chunks for file formats & Centralize logic for different auth configs (#11639) 
* Centralize Auth and File formats datalake
 2023-05-19 18:54:28 +05:30
Pere Miquel Brull
d52d773707
Send encrypted automation workflow (#11681) 2023-05-19 15:04:42 +02:00
Mayur Singal
e9992a52a8
Fix #1604: Add Spline Pipeline Connector (#11562) 
* Fix #1604: Add Spline Connector

* Add tests & grammer validation

* Spline UI Changes & Docs

* fix pipeline workflow doc

* chore: use common field for dbService name

* chore: use const for beta services

* chore: add service icon

* Update ingestion/src/metadata/ingestion/source/pipeline/spline/metadata.py

Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
 2023-05-19 14:46:32 +05:30
Pere Miquel Brull
50ad38ea0f
Fix #11548 - Secrets Managers comms with OMeta (#11602) 
* Remove secretsManagerCredentials from backend

* Remove secretsManagerCredentials from backend

* Add secrets manager loader

* Load SM in the ometa client

* Fix tests

* Fix tests

* Fix Lint

* Mock AWS region

---------

Co-authored-by: Ayush Shah <ayush@getcollate.io>
 2023-05-19 09:43:11 +02:00
Pere Miquel Brull
4626363fd8
Fix parsing for Storage (#11663) 2023-05-19 09:36:44 +02:00
Pere Miquel Brull
b60f198674
Fix GitHub reader headers and add docs (#11675) 
* Fix header and add docs

* Format
 2023-05-19 09:36:30 +02:00
Chirag Madlani
763b96b8fe
bump(ui): rjsf to v5 (#11549) 
* bump(ui): rjsf to v5
It will fix validation and extra parameter issues

* address comments
fix code smells
fix cypress failure

* revert schema files

* bigquery fix
sonar cloud fix

* fix big query

* fix superset

* fix cypress

* fix cypress for big query

* fix failures

* fix java issue

* revert connectionBasicType.json
create custom SupersetConnection.json

* fix Tableau service issues #11637
 2023-05-18 08:03:54 -07:00
Pere Miquel Brull
8795337f88
Clean NER Scanner imports (#11653) 2023-05-18 12:53:22 +02:00
Mayur Singal
e4997c3749
Fix #11571: Support custom database name for glue (#11631) 2023-05-18 14:16:56 +05:30
Onkar Ravgan
e6a7ffc57c
Fixes: 11540 Added api_version and domain fields to Salesforce (#11618) 
* Added sales new fields

* fixed image after merge

* Fix Failing Test

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
 2023-05-18 10:53:44 +05:30
Pere Miquel Brull
1b90badd0e
Restructure PII processor (#11640) 
* Restructure PII processor

* Restructure PII processor

* Format
 2023-05-17 15:58:17 +02:00
Mayur Singal
ea615eed9d
Fix elasticsearch error (#11635) 2023-05-17 15:48:20 +05:30
Pere Miquel Brull
c0ea11a477
Add libsasl deps (#11613) 2023-05-16 14:32:48 +02:00
Mayur Singal
b53a362772
Increase delay for system metric in e2e tests (#11607) 2023-05-16 06:57:36 +00:00
Pere Miquel Brull
019014b8d3
11579 Task - Add () in sample data (#11591) 2023-05-15 10:09:57 +02:00
Onkar Ravgan
3d9d4416b7
Fixed incompatible column name for Postgres version 11.6 (#11536) 
* postgres col name on version

* Added dependancy

* Added paranthesis validation

* review comments and tests
 2023-05-15 11:48:03 +05:30
Akash Jain
92d5bfa94e
fix: versions in main branch (#11478) 
* fix: versions in main branch

* Prepare main branch for next release 1.1.0

* prepare main for latest release 1.0.1
 2023-05-15 10:42:29 +05:30
Mayur Singal
e61bb3cf0d
Fix snowflake E2E: add system metric sleep time (#11569) 2023-05-12 14:09:57 +05:30
Onkar Ravgan
cff403a05a
Validate if tags are created before attaching them to CreateRequest (#11554) 
* Added tags validation

* typo fixed
 2023-05-11 16:04:55 +00:00
Mayur Singal
ef7b02529d
Remove sqlalchemy-ibmi from db2 dependency (#11553) 
* Remove sqlalchemy-ibmi from db2 dependency

* remove from json schema

* add migration

* update what's new
 2023-05-11 15:03:26 +02:00
Pere Miquel Brull
6040078a7c
Validate context in Airflow (#11551) 2023-05-11 12:23:43 +02:00
Pere Miquel Brull
f22d604c54
Remove old tests (#11505) 
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
 2023-05-11 10:29:30 +02:00
Sriharsha Chintalapani
e3a99e2f84
Fix Airflow ingestion: check if downstream task are null (#11539) 
* Fix Airflow ingestion: check if downstream task are null

* Fix Airflow ingestion: check if downstream task are null
 2023-05-11 06:43:57 +02:00
Mayur Singal
f7a0d3f5f2
Fix E2E Vertica & System Metric (#11525) 2023-05-10 17:35:55 +05:30
Mayur Singal
be7b056d72
Fix tableau models to make fields optional (#11513) 2023-05-10 12:23:44 +05:30
Teddy
60de33d7cf
Fixes #11384 - Implement mem. optimization for sys. metrics (#11460) 
* fix: optimize system metrics retrieval for memory

* fix: ran python linting

* fix: logic to retrieve unique system metrics operations

* fix: added logic to clean up query before parsing it

* fix: added E2E tests for rds, bq, snflk system metrics

* fix: ran python linting

* fix: fix postgres query + add default byte size to env var

* fix: ran python linting
 2023-05-09 12:05:35 +02:00
Pere Miquel Brull
80af2b20bb
Centralize SSL Schemas & Add SSL for Airflow REST (#11363) 2023-05-08 21:05:10 +02:00
Ayush Shah
65fc94a292
Fix Dynamodb connection (#11473) 
---------

Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Nahuel Verdugo Revigliono <nahuel@getcollate.io>
 2023-05-08 15:48:38 +00:00
Mayur Singal
ddc3060184
Fix vertica comments for version 9 (#11479) 2023-05-08 11:04:28 +00:00
Mayur Singal
074418eac3
Fix: Snowflake support extarnal tables (#11466) 2023-05-08 15:54:42 +05:30
Mayur Singal
a897954afc
Improve filtering for lineage query (#11457) 2023-05-08 15:18:51 +05:30
Ayush Shah
2c9ba537eb
Fix min max on rowversion/timestamp mssql (#11455) 2023-05-08 14:52:53 +05:30
Mayur Singal
e20f75fa54
Fix #11468: Fix salesforce scheme issue (#11474) 2023-05-08 14:02:17 +05:30
Pere Miquel Brull
aaf92a9ae8
Bump python-daemon (#11477) 2023-05-08 09:59:57 +02:00
Nahuel
b22192575d
Fix#11216: Use Enum for SASL Mechanism for Kafka and Redpanda connections (#11385) 
* Fix#11216: Use ENUM for SASL Mechanism for Kafka and Redpanda connections

* Minor change

* Address PR comments

* Fix postgres migration
 2023-05-08 09:35:49 +02:00
Noe Alejandro Perez Dominguez
b9aef4156f
Allow client requests to be retried (#11464) 
This is regarding: https://github.com/psf/requests/issues/4664

The change allows errors to be handled and retried.

Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
 2023-05-08 08:31:55 +02:00
Keith Sirmons
65c5b44eaa
Impala Connection Profiler is_nan rollback; Histogram fix. (#11388) 2023-05-05 21:45:30 +02:00
Onkar Ravgan
c7902344cb
Added tableau owner test connection step (#11436) 
* Added tableau owner test connection

* typo fixed

* remvoed generic exception
 2023-05-05 18:29:50 +05:30
Mayur Singal
2793c927cd
Fix postgres cypress issue (#11441) 2023-05-04 17:09:27 +00:00
Mayur Singal
6ddf7034ae
Add exception handling for tableau sheets (#11429) 
* Add exception handling for tableau sheets

* remove meating
 2023-05-04 16:38:27 +05:30
Mayur Singal
3f65c8ab0f
Fix Vertica column fetch query (#11424) 
* Fix vertica column fetch query

* fix timestamp datatype
 2023-05-04 13:37:00 +05:30
Mayur Singal
186eb252d7
Fix #11332: Fix databricks sample data ingestion for array datatype (#11420) 
* Fix #11332: Fix databricks sample data ingestion for array datatype

* Fix checkstyle
 2023-05-04 13:36:12 +05:30
Mayur Singal
49f3bae15e
Fix gemetry type for postgres (#11394) 2023-05-04 13:02:50 +05:30
Teddy
0a7f114281
fix: added logic to handled tests with no in result (#11409) 2023-05-03 21:59:23 +02:00
NiharDoshi99
02d4a1d7d6
making downstream_task_ids field optional for airflow AirflowDagDetails (#11405) 
* making downstream_task_ids field optional for airflow AirflowDagDetails

* update requirements file for airflow
 2023-05-03 22:09:10 +05:30
Onkar Ravgan
7e9c02fe6f
Fixed clean_query method for \n (#11389) 
* Fixed clean query method

* fixed regex and tests

* updated regex
 2023-05-03 18:08:54 +05:30
Keith Sirmons
00289bd85f
Fixes#11189: Implement Impala and hive get_view_definition (#11237) 
* updated metadata to work with the impala query engine.
Uses the describe function to grab column names, data types, and comments.

* added the ordinalPosition data point into the Column constructor.

* renamed variable to better describe its usage.

* updated profile errors.
Hive connections now comment columns by default.

* removed print statements

* Cleaned up code by pulling check into its own function

* Updated median function to return null when it is being used for first and third quartiles.

* updated metadata to work with the impala query engine.
Uses the describe function to grab column names, data types, and comments.

* added the ordinalPosition data point into the Column constructor.

* renamed variable to better describe its usage.

* updated profile errors.
Hive connections now comment columns by default.

* removed print statements

* Cleaned up code by pulling check into its own function

* Updated median function to return null when it is being used for first and third quartiles.

* removed print statements and ran make py_format

* updated to fix some pylint errors.
imported Dialects to remove string compare to "impala" engine

* moved huge comment into function docstring.
This comment shows us the sql to get quartiles in Impala

* added cast to decimal for column when running average in mean.py

* fixed lint error

* fixed ui ordering of precision and scale.
Precision should be ordred in front of scale since the precision is set first in decimal data types

* Added get_view_definition to hive and impala connectors.

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
 2023-05-03 15:06:33 +05:30
Nahuel
3ba29e7f0e
Fix: Redshift E2E tests (#11396) 2023-05-03 08:12:32 +05:30
Nahuel
1ec6e5e285
Fix#11311: Add IBM dependency for i Series in DB2 connector (#11381) 2023-05-02 15:50:39 +02:00
Teddy
f8c667b504
Fix median for concatenable types (#11382) 
* fix: median/fq/tq for concatenable types

* fix: ran linting
 2023-05-02 10:45:26 +00:00