2916 Commits

Author SHA1 Message Date
Pere Miquel Brull
b1a4faf171
MINOR - Metadata Actions schemas (#15822)
* MINOR - Metadata Actions schemas

* add type for left side menu bar items

* fix the required fields in automatorAppConfig.json schema

* update constants

* mixin

* create new styled component to show in alerts form card

* Add new translations for ingestion pipeline and location

* Fix import statement in AlertDetailsPage.tsx

* Update automatorAppConfig.json schema to use an array for the 'type' property

* config

* move the advanced search configs to a new classBase for better customization

* Update automatorAppConfig.json schema to add "name" and "description" fields

* Update automatorAppConfig.json schema to remove "name" and "description" fields

* Add new translations for description-plural in multiple language files

* filter Automator

* revert remove

* Update pipeline actions interface and component to handle edit click event

* Update select type for advanced search fields in AdvancedSearchClassBase.ts

* Update ErrorPlaceHolderIngestion component to accept cardClassName prop

* Update FormCardSection component to accept className prop

* Update TagsContainerV2 component to include state property in tag data

* Update TagsContainerV2 component to include defaultState and defaultLabelType properties

* Update ingestionPipelineAPI.ts to include applicationType parameter in getIngestionPipelines function

* Update AdvanceSearchProvider.test.tsx to include pathname in useLocation mock

* work on comments

* fix sonar lint errors

* format

* Update useLocation mock in AdvanceSearchProvider.test.tsx to include pathname

* revert remove

* Update LogsViewerPage interfaces and component to include application_task in IngestionPipelineLogByIdInterface

* Update LeftSidebar component to import LeftSidebarItem interface and fix type annotation in children map function

* fix observability alert test

* revert the form list add method change

---------

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2024-04-19 12:46:58 +02:00
Ayush Shah
d5b1465406
Fixes #14113 - Allow SSL file uploads (#15828) 2024-04-19 11:38:27 +05:30
Imri Paran
247e05a83d
MINOR: Update README.md python version (#15893) 2024-04-18 21:33:16 +05:30
IceS2
fd51df25fa
Removed stale examples. Update dependencies (#15951) 2024-04-18 17:41:09 +02:00
Mayur Singal
18da8a5964
Fix #15208: Lineage - log query parsing issue summary (#15921) 2024-04-18 19:07:51 +05:30
Mayur Singal
d81978c388
MINOR: Fix sample data CI error (#15934) 2024-04-18 14:18:22 +05:30
harshsoni2024
39fc846fdb
Fix #9281: Column level lineage for other entities (#15827) 2024-04-18 12:20:33 +05:30
Ayush Shah
0952e4ff81
Fix All_ table to DBA_ (#15937) 2024-04-18 10:09:15 +05:30
Suman Maharana
6915c1a1b6
Fix #15090 [1.4] Added dbt existing glossary and tier ingestion (#15833)
* Added dbt glossary and tier ingestion

* changed elif to if condition:pylint

* Added dbtMeta model for glossary and tier

* removed source

* Optimization: Reused the dtb_tags

* pylint format

* py_format

* Removed unnecessary models and minor changes

* Removed empty string assignment
2024-04-17 17:10:20 +02:00
Imri Paran
47f0d99333
MINOR: add raise_from_status for sql_server test (#15931)
* Update test_metadata_ingestion.py

* Update test_metadata_ingestion.py

* fixed import
2024-04-17 14:52:10 +02:00
Imri Paran
29cd58b628
MINOR: added integration test for SQL SERVER (#15919)
* adventure works mssql test case

* adventure works mssql test case

* fixed tests

* fixed tests

* fixed tests

* fixed tests
2024-04-17 12:19:37 +02:00
Mayur Singal
67f01625b0
MINOR: Fix Glue Sample Data (#15922) 2024-04-17 10:43:36 +05:30
IceS2
2bfc4c450d
MINOR: Add mapping for xid and bpchar for postgresql (#15912)
* Add mapping for xid and bpchar for postgresql

* fix checkstyle
2024-04-16 15:40:46 +02:00
Mayur Singal
269ed6d5bb
MINOR: Fix databricks connector (#15894) 2024-04-15 16:18:58 +05:30
Mayur Singal
9439f58bef
MINOR: Add Databricks ssl dependencies (#15895) 2024-04-15 15:37:07 +05:30
Imri Paran
706d1ab97e
fixed ingestion of sample data for failed sample rows (#15879) 2024-04-15 07:59:27 +02:00
Imri Paran
8447cf0192
Update tests_mixin.py (#15874) 2024-04-12 10:00:35 +02:00
Teddy
4c4b7c67a4
Minor fix row computation for views (#15872)
* fix: row computation for views

* style: ran python linting

* fix: MySQL innodb row count

* style: ran python linting
2024-04-11 16:55:39 +02:00
Ayush Shah
0c3e580592
MINOR: Fix Entity Link Error (#15864) 2024-04-11 18:41:59 +05:30
Ayush Shah
aeaaea4720
Modify and add threshold as status (#15866) 2024-04-11 18:39:46 +05:30
Pere Miquel Brull
a1404e6b4a
MINOR - Clean ingestion dependencies (#15679)
* WIP - MINOR - Clean ingestion dependencies

* test

* test

* Clean imports

* add pyiceberg for test

* Revert "add pyiceberg for test"

This reverts commit ab26942736586f089a57a644ffd727aca200db62.

* add pyiceberg for test

* Remove docker dep

* clean local docker sh

* MINOR - AKS Airflow troubleshooting docs

* Fix action

* clean local docker sh
2024-04-11 14:30:40 +02:00
Imri Paran
b2ce491ff1
MINOR: Add failed rows sample to test case (#15682)
* add failed sample data

* format

* fixed masking pii data in test failed rows sample

* format

* failedRowsSamples -> failedRowsSample

* failedRowsSamples -> failedRowsSample

* fixed tests

* format

* wip

* added computePassedFailedRowCount to python client

* comment for loggerLevel

* format

* fixed tests

* tests for putting / deleting failed samples

* format

* format

* added test case for pii test

* changed method name to deleteTestCaseFailedRowsSample

* added getComputePassedFailedRowCount
2024-04-10 17:00:00 +02:00
harshsoni2024
c671e64f69
[MINOR] Fix tableau e2e (#15824) 2024-04-08 21:17:37 +05:30
IceS2
c909ff8857
MINOR: Fix e2e tests (#15829)
* Update values

* Update values

* Fix checkstyle
2024-04-08 15:58:32 +02:00
IceS2
12a4c578a2
MINOR: Fix jsonpatch operation order (#15680)
* Mantain the OperationType Order when considering the dividing groups

* Remove reordering the jsonpatch operations from the backend

* Fix checkstyle

* Fix UnitTests to comply with no reordering

* Initial idea on how to fix our current jsonpatch builder from python

* fix(JsonUtils): Change JSONPatch library used

When creating a JSONPatch by using the 'createDiff' method, the library
we are using is not returning a correct JSONPatch when removing multiple
items from an array.

Since the library doesn't provide good ways to override this behavior
and fix it, we decided to move away from it and use the json-patch
library only for this specific operation.

* Fix linters

* Add docstrings

* Refactor patch updated on ingestion framework

* Add UnitTests

* Fix linters
2024-04-05 15:52:01 +02:00
IceS2
47a70a4fd7
MINOR: Fix OMetaLifeCycleData call (#15813)
* Fixx OMetaLifeCycleData call

* Fix linters
2024-04-05 07:26:29 +02:00
Onkar Ravgan
370abda43f
dbt docs revamp (#15787) 2024-04-03 20:42:30 +05:30
Suman Maharana
16eaf925e9
FIX #13553 Added option to exclude drafts: superset ingestion (#15770)
* Added option to exclude drafts: superset ingestion

* Updated supserset yaml docs

* Added tests for exlcude draft dashboards

* Added tests for exlcude draft dashboards

* Formatted queries.py
2024-04-03 17:07:02 +05:30
Ayush Shah
b79e5c064b
Fix 15576 - Eval Data Type issue fix (#15702) 2024-04-03 15:51:19 +05:30
Teddy
205850be79
[MINOR] fix antlr parser definition for entity link (#15758)
* fix: update antlr regex for entity fqn

* fix: update antlr rule to allow single character

* style: ran python linting

* fix: updated antlr token for NAME_OR_FQN
2024-04-03 08:34:43 +00:00
harshsoni2024
feb33a0cc2
Fix #12964: Qlik Sense & Qlik Cloud filter draft dashboards (#15726)
* Fix #12964: filter draft dashboards from config

* Fix #12964: add unit test for qlik_sense

* Fix #12964: added UI and doc code

* Fix #12964: move includedraftdashboard flag from source_connection to source_config

* Fix #12964: filter draft dashboards in qlikcloud

* Fix #12964: add unit test for qlik cloud

* Fix #12964: remove unnecessary comments, code clean

* Fix #12964: pylint changes
2024-04-02 14:30:33 +02:00
Ayush Shah
7c7a7c35a9
Check Test Connection Oracle DBA Access (#15764) 2024-04-01 18:18:21 +05:30
Onkar Ravgan
2e640f6ef2
fixed dbt exception condition (#15762) 2024-04-01 15:21:06 +05:30
Onkar Ravgan
2931798be9
fixed superset class init (#15761) 2024-04-01 15:10:08 +05:30
Imri Paran
5b963a5d9e
MINOR: remove metadata/__init__.py (#15750) 2024-03-29 15:15:52 +05:30
harshsoni2024
884029f031
Fix #14089: Add QlikCloud connector support (#15610) 2024-03-28 20:15:22 +05:30
Pere Miquel Brull
890820ed92
MINOR - App routes & datamodel (#15722)
* MINOR - App routes & datamodel

* fix future annotations

* fix future annotations
2024-03-27 19:12:24 +01:00
Mayur Singal
bf0ec44f4b
Fix #15678: Accomodate Metabase API changes (#15692) 2024-03-27 15:19:20 +05:30
Mayur Singal
6acb47a87a
MINOR: Fix unity catalog test connection (#15712) 2024-03-27 15:14:26 +05:30
IceS2
ce3f124a33
MINOR: Add Multithread Documentation (#15706)
* Add Multithread Documentation

* Add general considerations
2024-03-27 08:30:16 +01:00
Mayur Singal
0532bb1226
MINOR: Fix external table lineage processing (#15713) 2024-03-27 12:37:20 +05:30
Mayur Singal
8073a80989
Fix #14285: Add column lineage support for tableau datamodels (#15646) 2024-03-27 11:03:40 +05:30
Pere Miquel Brull
9d7bfa363e
MINOR - Clean metadata CLI (#15631)
* Docs

* MINOR - Clean metadata CLI

* remove tests
2024-03-26 16:36:47 +01:00
Ayush Shah
6039fe9462
Fix TypeError Missing arg (#15698) 2024-03-26 18:32:32 +05:30
Mayur Singal
6b90c245d4
MINOR: Add support for json schema parsing for datalake & s3 (#15615) 2024-03-26 10:03:21 +05:30
IceS2
e7c9d6aa7f
FIXES 15215: Implement initial Multithreading approach for the Metadata Ingestion on Databases (#15130)
* Implement Initial MultiThread suggestion

* Update all the ingestion sources to use the new ContextManager

* Fix missing wraps on decorator

* Fix Unittests

* Fix linters

* Fix linters

* Fix BigQuery UnitTests

* Add UnitTests to the newly created code

* Fix unittest

* change the threads from table to schemas

* Update README.md

* Small change suggested by Sonar

* Slight change to test a different way to multithread over tables

* Debug changes

* More multithread tests

* Remove uneeded wait time

* Testing

* refactor code based on removal of time.sleep

* Fix wrong paste

* Improve ExecutionTimeContextManager

* Fix missing .get() and unit tests

* Fix conflicting changes

* Update Multithread logic with the incremental extraction

* Fix linters

* Fix unittest

* Remove commented code

* Fix Unittests

* Fix checkstyle

* Change default to threads = 1
2024-03-25 18:20:40 +01:00
Ayush Shah
00677a1e1b
Fix External Account Json Schema Issue (#15671) 2024-03-23 16:47:55 +05:30
Pere Miquel Brull
a79e79ef3d
#15662 - List All test cases from a table in DQ (#15665)
* #15662 - List All test cases from a table in DQ

* #15662 - List All test cases from a table in DQ

* #15662 - List All test cases from a table in DQ
2024-03-22 11:30:02 +01:00
Ayush Shah
8b880bbf91
Fixes 14370: Add Azure Client, support Default Creds (#15554)
* Add Azure Client, support Default Creds
2024-03-22 14:28:42 +05:30
Mayur Singal
ad28af4f4f
MINOR: Fix sample data upload - binary data error (#15659) 2024-03-22 12:13:26 +05:30