3385 Commits

Author SHA1 Message Date
Mayur Singal
7197356101
Minor: Fix pyarrow import error (#15004) 2024-02-02 16:28:40 +05:30
Onkar Ravgan
131e5f5767
Modified snowflake view def query (#14994) 2024-02-02 14:34:43 +05:30
Onkar Ravgan
25c6260bd7
MINOR: Optimised dbt topology and better exception handling for athena LF tags (#14978)
* dbt top and athena excp

* Added dbt excp handling

* sourcehash in patch
2024-02-02 09:21:15 +01:00
Ayush Shah
a4f3ed8eec
Minor: Fix E2E Powerbi (#14991) 2024-02-01 19:31:38 +05:30
Akash Jain
7a90de2e9e
fix(docker): K8s requires apache-airflow-providers-cncf-kubernetes pip package (#14989) 2024-02-01 18:20:45 +05:30
Teddy
9a4a9df836
Fix #14895 - Get Metadata from Parquet Schema (#14956)
* linting: fix python linting

* fix: get column types from parquet schema for parquet files

* style: python linting

* fix: remove displayType check in test as variation depending on OS
2024-02-01 09:02:52 +01:00
Sriharsha Chintalapani
2e95fcb98d
Fix #14786: Suggestions API (#14821)
* Fix #14786: Suggestions API

* Handle suggestions in ometa

* Minor: Optimise Databricks Client (#14776)

* MINOR - Fix SP topology context & Looker usage context (#14816)

* MINOR - Fix SP topology context & Looker usage context

* MINOR - Fix SP topology context & Looker usage context

* Fix tests

* Fixes #14598: Fix Tags / Labels ingestion on includeTags as False (#14782)

* fix(ui): password error message for char limits (#14808)

* fix(ui): password error message for char limits

* fix java side code

* Fixes #13556: Support for Salesforce table description ingestion (#14733)

* ISSUE-13556: Add suport for Salesforce table description ingestion

* ISSUE-13556: Remove unnecessary blank line

* ISSUE-13556: Fix to get description for each table

---------

Co-authored-by: Teddy <teddy.crepineau@gmail.com>

* MINOR - Better handling of Ingestion Pipeline Status (#14792)

* MINOR - Better handling of Ingestion Pipeline Status

* format

* format

* MINOR: Added table validation for cost analysis data (#14793)

* Added validation for cost analysis source

* centralized life cycle logic

* CYPRESS: simplify side navigation click in cypress (#14818)

* simplify side navigation click in cypress

* make sidbar item uses common enum

* fix cypress failure of outside import

* fix(#14326): tier dropdown is not working in advance search (#14780)

* improvement in advance search based on custom property

* fix a reading undefined property issue

* wip: advance search based on tier

* some code cleanup and improvement

* some fixes

* fix: ui flicker when advanceSearched is apply and refresh the page

* some cleanup

* no need to call customproperty api call, if entity not suppport customProperties

* minor change

* fix: autocomplete not working in tier search option in advance search modal

* added unit test for advance search provider component

* some cleanup

* added testcase for open modal

* added testcase for resetAllFilters method

* removed unwanted code

* added e2e test for testing tier advance search

* fix: e2e search flow for single field

* fix: string field not working after giving listValues in TierSearch

* fix: group query e2e test fix

* used asyncFetch way to get the tierOptions synchronously

* some cleanup

* remove unwanted lines

* some cleanup

* fix: selected option show option value instead of option title

* fix(minor): update skip icon for executions (#14809)

* Fixes #14803: ignore capitalization when confirming deletes  (#14804)

* ignore case when confirming deletes

* Test confirmation of deletes works when case differs 

Added test case for 'delete' as the confirmation text.

* minor(config): update openmetadata-ui code reviewers (#14823)

* Add Tests

* Add list/accept/reject apis

* initial ui changes

* localisation

* show suggestion for empty description

* ui feedbacks

* Fix permission check for entities without owner

* Fix entityLink and add tests

* Add update suggestion WIP

* Fix test

* Fix PUT and Pagination

* Fix styling

* update test

* Update status

* add OM server connection in apps

* add permissions check

* Fix CI

* Remove TODO

* Fix feedResourceTest

* fix unit tests

* add private configs for apps

* add private configs for apps

* fix update application icons

* minor center align icon

* add private configs for apps

* Format

* Fix pydantic gen

* Remove token

* Update name

* Rework private conf

* Fix apps

* Fix apps

* Format

* Format

* show metapilot only if its installed

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: kwgdaig <18678754+kwgdaig@users.noreply.github.com>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: Abhishek Porwal <80886271+Abhishek332@users.noreply.github.com>
Co-authored-by: Carlo Q <carlo@machina.bio>
Co-authored-by: karanh37 <karanh37@gmail.com>
2024-01-31 18:51:09 -08:00
Ayush Shah
0d66c0e514
Minor: Add extended glossary (#14918)
* Add Nested glossary to extended sample data

* Fix lint

* Add Stress test config

* Fix Lint
2024-01-31 07:13:04 -08:00
Onkar Ravgan
4c537f2d68
MINOR: Fix E2E failures (#14940)
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2024-01-30 14:05:51 +05:30
Teddy
4534945e92
fix #14705: add fall back method to compute metric without sum if overflow (#14896)
* fix: add fall back method to compute metric without sum if overflow

* linting: fix python linting
2024-01-29 09:01:48 -08:00
Pere Miquel Brull
1492f79da1
MINOR - Revert #14468 - DuckDB Connector (#14934) 2024-01-29 16:10:53 +01:00
IceS2
373cafcda2
Fixes #5448: Implement initial Iceberg Connector using PyIceberg (#14825)
* Create the iceberg connection schema

* Link the IcebergConnection configuration with the forms on the UI

* Add the pyiceberg dependency on the ingestion package

* Create the get_connection and test_connection functions

* First iteration on the iceberg ingestion logic

* Add A more comprehensive implementation of the Iceberg Source

* Add UnitTests

* Update icebergConnection definition

* Update the iceberg souce code based on new schema

* Updated icebergConnecgtion schema for simplicity and to be able to configure Converters

* Updated setup dependencies to be more flexible

* Updated get_owner_ref logic

* Fix formatting

* Changed the icebergConnection json schema structure to enable the ClassConverters

* Add the IcebergCatalog and IcebergFileSystem ClassConverters

* Refactor the code to take into account the new jsonSchema structure

* Fix formatting

* Add Documentation for the Iceberg Connector

* Fix Menu order for Iceberg

* ui: add Iceberg service icon and constant

* Fix DynamoDb Catalog issue due to how PyIceberg instantes it

* Changed uri title to URI

* Fix ClassConverter for Iceberg

* Fix GetSecretValue for password types

* Fix formatting

* Fix formatting

* Add Iceberg Connector Images for the docs

* Add pylint disable for Hacky super() call

* Add Iceberg.md for the UI docs

* Fix pylint complaint

* Fix pylint complaint

* Fix UnitTests

* fix type error and unit tests

* update pipeline type checks

* Fix Sonar Cloud complaints

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-01-29 06:32:58 +01:00
Pere Miquel Brull
ab6c175711
MINOR - Fix Python CI (#14905)
* #13934 - Update test connection no response error msg

* #13934 - Update test connection no response error msg

* MINOR - Fix python CI
2024-01-28 23:52:09 +01:00
Pere Miquel Brull
0332fb9fe5
#13934 - Update test connection no response error msg (#14897)
* #13934 - Update test connection no response error msg

* #13934 - Update test connection no response error msg
2024-01-28 10:25:15 -08:00
Pere Miquel Brull
db985fda57
MINOR - Snowflake system queries to work with ES & IDENTIFIER (#14864) 2024-01-26 18:41:16 +05:30
Pere Miquel Brull
6a6770042c
MINOR - Update step names (#14875)
* MINOR - Update step names

* Update e2e regex

* Format
2024-01-26 12:59:54 +01:00
Marco Rezende
37d967013b
ISSUE-14742: Adjusting metadata.py to filter by enum value (#14884) 2024-01-26 07:30:05 +01:00
NiharDoshi99
c1d62186df
MINOR - metadata tag extraction for Databricks (#14874)
* metadata tag extraction for databaricks

* fix python test

* changes as per comment

* fix python test

* fix python checkstyle
2024-01-26 07:09:24 +01:00
Mayur Singal
20605cf7c7
MINOR: Snowflake fix lineage & usage muti db query (#14866) 2024-01-25 15:28:46 +05:30
Ayush Shah
1552aeb2de
Fix #13149: Multiple Project Id for Datalake GCS (#14846)
* Fix Multiple Project Id for datalake gcs

* Optimize logic

* Fix Tests

* Add Datalake GCS Tests

* Add multiple project id gcs test
2024-01-25 10:52:16 +01:00
Mayur Singal
75471cfba9
MINOR: Fix Databricks pipeline client pagination (#14860) 2024-01-25 14:41:25 +05:30
Pere Miquel Brull
85e2058979
MINOR - Fix & Organize topology context (#14838)
* MINOR - Fix & Organize topology context

* Handle missing context charts
2024-01-25 08:22:07 +01:00
Onkar Ravgan
80fff72949
Fix #14794: Refactored and cleaned owner processing in sources (#14817)
* refactor owner processing

* Add exception handling and fix pytest

* review comments addressed

* looker tests

* fixed pycheckstyle
2024-01-25 06:46:22 +01:00
Ayush Shah
e30ffd71ce
Fix 13970: Fix Consecutive Array issue (#14852) 2024-01-25 10:38:56 +05:30
Onkar Ravgan
d1460b675b
Add condition to fix error (#14856) 2024-01-25 10:18:01 +05:30
Mayur Singal
17fb2cabca
MINOR: Lineage handle copy queries being skipped (#14855) 2024-01-25 10:15:32 +05:30
NiharDoshi99
2efa0c9e28
#13974 handle for hyphen in schema and median function (#14834) 2024-01-24 15:57:36 +05:30
Teddy
d228a93fbf
fix: increase floating point precision (#14827) 2024-01-24 09:19:19 +01:00
Shiyang Xiao
9f5a70bd71
MINOR - update docs & added unit test for SAS Connector (#14743)
Co-authored-by: Shiyang Xiao <Shiyang.Xiao@sas.com>
2024-01-23 14:55:29 -08:00
Onkar Ravgan
3cf8e0a308
MINOR: Added table validation for cost analysis data (#14793)
* Added validation for cost analysis source

* centralized life cycle logic
2024-01-23 09:55:24 +01:00
Pere Miquel Brull
8f2c2b21c2
MINOR - Better handling of Ingestion Pipeline Status (#14792)
* MINOR - Better handling of Ingestion Pipeline Status

* format

* format
2024-01-23 09:24:19 +01:00
kwgdaig
5d5c384dec
Fixes #13556: Support for Salesforce table description ingestion (#14733)
* ISSUE-13556: Add suport for Salesforce table description ingestion

* ISSUE-13556: Remove unnecessary blank line

* ISSUE-13556: Fix to get description for each table

---------

Co-authored-by: Teddy <teddy.crepineau@gmail.com>
2024-01-23 07:36:36 +00:00
Ayush Shah
fda74d28b3
Fixes #14598: Fix Tags / Labels ingestion on includeTags as False (#14782) 2024-01-23 11:58:23 +05:30
Pere Miquel Brull
337796d612
MINOR - Fix SP topology context & Looker usage context (#14816)
* MINOR - Fix SP topology context & Looker usage context

* MINOR - Fix SP topology context & Looker usage context

* Fix tests
2024-01-23 07:02:39 +01:00
Mayur Singal
492bac32c0
Minor: Optimise Databricks Client (#14776) 2024-01-23 11:28:02 +05:30
Mayur Singal
eb1550205a
MINOR: Fix Vertica profiler median query (#14800) 2024-01-19 18:53:14 +00:00
NiharDoshi99
3f78e072e1
#13429 support for struct data type in hive (#14785) 2024-01-19 18:26:53 +05:30
Mayur Singal
4e02cb1c54
Fix #14764: Profiler make compute metric configurable (#14791) 2024-01-19 18:13:26 +05:30
Akash Jain
72fc0cf685
fix: Security Scan CI + Makefiles (#14790) 2024-01-19 14:18:07 +05:30
Onkar Ravgan
f10b5a3617
Fixed athena e2e db (#14774) 2024-01-19 11:51:38 +05:30
Teddy
9c4ec6c760
fix: added fall back to table metric computation (#14771) 2024-01-19 00:33:57 +05:30
Ayush Shah
831fce5b7e
Fixes 10709: Add useFqnForFiltering to profiler workflow (#14717) 2024-01-18 18:52:43 +05:30
Onkar Ravgan
c5c7171036
MINOR: Added Support for Bigquery and Redshift lifeCycle data (#14622)
* Added BQ and redshift lifeCycle

* added usage fixes
2024-01-18 15:36:43 +05:30
Ayush Shah
0d0d820305
MINOR: Add Static Sample Data, Modify Lineage (#14752) 2024-01-17 20:07:49 +05:30
NiharDoshi99
6620389ad1
fix null prop. when entire col is null (#14758) 2024-01-17 16:00:06 +05:30
Teddy
c90a86b8ad
chore: remove typing-extension dependency (#14757) 2024-01-17 09:58:10 +00:00
Mayur Singal
07ea5e97c8
Minor: Postgres cypress move to query log file usage (#14750) 2024-01-17 14:26:57 +05:30
NiharDoshi99
c120ae98a4
#14727 change all all_ query to dba_ query (#14748) 2024-01-17 12:38:42 +05:30
Vijay Ravi
abe716d7fa
MINOR: Update jsonpatch package version (#14740)
* MINOR: Update jsonpatch package version

* Format

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-01-17 07:17:11 +01:00
Pere Miquel Brull
eadda0e3f1
#9269 - Add MSSQL Stored Procedure Support (#14739)
* Remove unnecessary field

* Remove unnecessary field

* Support query comments in MSSQL

* Remove unnecessary field

* Format

* Add external type

* Add MSSQL SP support
2024-01-17 06:37:27 +01:00