3385 Commits

Author SHA1 Message Date
Mayur Singal
dccfd9a776
Minor: Add Column Support in Protobuf Parser (#14745) 2024-01-17 09:31:27 +05:30
Onkar Ravgan
f2219a10f3
Fixed oracle tests (#14738) 2024-01-16 17:39:10 +01:00
Onkar Ravgan
13b326d11c
Fix 14040: Part 3 Added updated field to status for patched entities (#14708)
* Added updated field to status

* Fixed e2e test
2024-01-16 17:53:05 +05:30
Onkar Ravgan
64a4e1afce
Fix 12180, 14158: Added LF tags to Athena (#14718)
* Added LF tags to athena

* fixed pytests

* Added docs
2024-01-16 14:24:31 +05:30
NiharDoshi99
54d34934c1
#14630 added oracle stored procedures (#14641) 2024-01-15 18:28:27 +05:30
Ayush Shah
709fa7415f
Fixes 13343: Fix Superset table name issue (#14714) 2024-01-15 18:19:27 +05:30
Ayush Shah
98eae7154a
Fix Airflow Source Url (#14712) 2024-01-15 12:31:43 +00:00
Mayur Singal
355f933ef0
Minor: Fix Superset API Test Connection (#14696) 2024-01-15 15:25:48 +05:30
Mayur Singal
21a701f360
Minor: Fix Postgres Cypress Failure (#14698) 2024-01-13 11:49:36 +05:30
Onkar Ravgan
f1049d55c4
Fixed mode test connection (#14693) 2024-01-12 18:29:12 +05:30
Mayur Singal
a7c17c5351
Fix #14653: Fix Unity Catalog no children struct issue (#14688) 2024-01-12 12:59:50 +05:30
Shiyang Xiao
241f3c68d7
Fixes #14413: Add SAS connector (#14415)
* feat: SAS Viya connector

* refactor SASCatalog to SAS

* add SAS logo to UI and connection documentation

* doc changes

* modify ingestion logic

* revert original changes

* added support for dataflow & perfect logic for reports/datatables

* add filter doc

* more updates to perfect ingestion for each asset type

* fix a bug with table lineage not created properly

* Delete ingestion/pipelines/sasCatalog.yaml

* precomit fix

* Conversion to database connector

* minor fixes

* make custom properties type generic

* Add SAS javaEnum

* add dummy variable for sas.yaml

---------

Co-authored-by: lizmc <liz.mcintosh@sas.com>
Co-authored-by: Shiyang Xiao <Shiyang.Xiao@sas.com>
2024-01-11 06:46:57 -08:00
Mayur Singal
66d92300c6
Fix #13894: Ingest only open search indexes ES (#14685) 2024-01-11 18:23:49 +05:30
Ayush Shah
9c6d202555
Add Sample data, modify regex pattern (#14467) 2024-01-11 14:23:33 +05:30
Mayur Singal
190212c8ac
Fix #11556: Add support for Db2 for IBM i (#14680) 2024-01-11 12:35:52 +05:30
Onkar Ravgan
aee42cf72b
removed patch warnings (#14669) 2024-01-10 23:02:30 +05:30
Mayur Singal
810e1df051
Minor: Fix Databricks Pipeline Test Connection Fix (#14667) 2024-01-10 18:37:00 +05:30
Mayur Singal
8197fe56ec
Minor: Snowflake - Add Support For Materialised Views (#14664) 2024-01-10 18:36:42 +05:30
Onkar Ravgan
d913586622
Fixed auth header in rest client (#14632) 2024-01-09 11:47:09 +05:30
Mayur Singal
3b61938c6d
Minor: Fix Mysql E2E Count (#14631) 2024-01-09 11:28:43 +05:30
Pere Miquel Brull
24643a397a
#14492 - Fix Snowflake SP parsing with empty signature (#14623) 2024-01-08 11:16:35 -08:00
Pere Miquel Brull
e8eb2bda3c
MINOR - Hide AutoTagger, add MetaPilot search, and fix apps status calls (#14201)
* MINOR - Hide AutoTagger & MetaPilot search & Fix apps status calls

* Fix method call

* Fix method call

* Fix method call

* "additionalProperties": true

* Use EntityRef

* Use ES sink

* Update UI schema type
2024-01-08 14:57:07 +01:00
Pere Miquel Brull
b92fd5cde4
MINOR - Move test case incident to the top of the data_quality_data_time_series (#14600)
* Add field and index

* MINOR - Move test case incident to the top of the data_quality_data_time_series table

* Fix test

* Fix compile

* Format

* Update incidentId

* Rename field

* Fix patch
2024-01-08 07:42:15 +01:00
Onkar Ravgan
ecdb7b9f41
Fixes 14109 and 14325: Optimised Tableau Connector (#14548)
* Optimised tableau conn

* Added comment
2024-01-08 06:33:05 +01:00
Mayur Singal
c3903b3f45
Minor: Fix IBM DB2 Depenency Installation Issue (#14613) 2024-01-08 10:18:04 +05:30
Ayush Shah
d513dc4d4d
#11890: Add GCS Json schema changes (#14559)
* Issue-11890: Add GCS Json schema changes

* Modify Gcs Security config

* Add Test Def for gcs

* Add gcs for storage_metadata_config
2024-01-06 19:27:36 -08:00
Mayur Singal
a789fc86d6
Fix #13053: Remove Connection URI config MongoDB (#14584)
* Fix #13053: Remove Connection URI cofig MongoDB

* pyformat & test fixes
2024-01-05 10:51:12 -08:00
Mayur Singal
d82770a91e
Fix #14421: Airflow - Handle pipeline status when dag killed (#14562) 2024-01-05 18:14:26 +05:30
Pere Miquel Brull
0255171218
MINOR - Create Test Case Resolution ts entry & delete resolution when… (#14541)
* MINOR - Create Test Case Resolution ts entry & delete resolution when Test Case is deleted
2024-01-05 09:15:49 +01:00
Mayur Singal
bf74ef9e22
Minor: Filter Create SP & Declare Queries for MSSQL Lineage (#14586) 2024-01-05 12:13:54 +05:30
Onkar Ravgan
7fdff15648
fixed dbt runresults ts (#14572) 2024-01-04 12:36:23 +05:30
Onkar Ravgan
a892267fbb
Fixed salesforce table req (#14570) 2024-01-04 10:45:58 +05:30
Mayur Singal
d197164125
Fix #14176: Add AzureSQL dialect in lineage (#14563) 2024-01-03 08:11:05 -08:00
NiharDoshi99
78bd8f98b5
#10203 delete temp creds files (#14536) 2024-01-02 13:09:15 +05:30
Cristian Calugaru
1631172528
Issue 10587 - Storage Service doesn't handle deleted containers (#14497)
* global manifest option for storage services

* added a no metadata config source option for global manifest s3 services option

* merge fixes

* more merge fixes.

* black stuff

* test fixes

* formatting

* storage service container automatic deletion support

* reverted auto-modified files.

* styling

* linting
2023-12-30 13:27:16 -08:00
mgorsk1
5ae83b284a
Fixes 14528: skip statuses for tasks that are no longer present in Airflow DAG (#14529)
* 🐛 Fix attribute name

* import AirflowTask model

* make py_format

* make py_format_check
2023-12-30 13:26:34 -08:00
Pere Miquel Brull
f4bbca3f72
MINOR - Clean topology & add tests (#14527)
* Clean topo

* Format

* Add tests

* Fix tests

* Merge main
2023-12-29 17:00:59 +01:00
Pere Miquel Brull
3613d0b18a
MINOR - Airflow Lineage Backend to verify the fingerprint (#14530)
* hash

* MINOR - Airflow Lineage Backend to verify the fingerprint
2023-12-29 16:36:28 +01:00
Takahiro Nakayama
7d05d9f50a
Fixes: GetTags-403 Permission denied on resource project None. (#14521)
* Fix "GetTags-403 Permission denied on resource project None."

Signed-off-by: Takahiro Nakayama <civitaspo@gmail.com>
2023-12-29 15:09:59 +05:30
Pere Miquel Brull
1c2219517e
#13649 - Add Pipeline Status Skipped (#14524)
* #13649 - Add Pipeline Status Skipped

* Format
2023-12-29 09:41:59 +01:00
Pere Miquel Brull
4b6034c527
#13638 - Filter task instance by removed state (#14515)
* Clean name logging and filter out removed tasks

* Update client logging

* Update client logging
2023-12-29 06:56:44 +01:00
Pere Miquel Brull
b84ce33b80
#11799 - Fix Airfow ownership & add pipeline tasks (#14510)
* Fix airflow owner and add tasks

* Add pipeline tasks ownership

* MINOR - Fix py CI

* Add pipeline tasks ownership

* Add pipeline tasks ownership

* MINOR - Fix py CI

* MINOR - Fix py CI

* Add pipeline tasks ownership

* patch team

* patch team

* Format
2023-12-28 10:25:00 -08:00
Mayur Singal
0c14a73fd1
Minor: Handle Column Lineage Exceptions (#14505)
* Minor: Handle Column Lineage Exceptions

* py format

* pylint
2023-12-27 13:04:10 -08:00
Teddy
61ef55290e
MINOR - generic profiler optimization for sampling and BQ (#14507)
* fix: limit sampling to specific column

* fix: handle bigquery struct columns

* fix: default partition to 1 DAY for BQ

* fix: default to __TABLES__ for BQ table metrics

* style: ran python linting

* style: fix linting

* fix: python style

* fix: set partition to DAY if not HOUR
2023-12-27 19:13:44 +01:00
Teddy
3dc642989c
Fixes #7729 - Add logic to compute passed/failed rows (#14472)
* feat: add test case resolution task workflow

* chore: add migration for test case resolution feature

* fix: removed required field for object compatibiity in older migrations

* fix: minor testCaseResolution status logic

* chore: revert migration for test case incident

* chore: update migration file

* style: renamed variables

* feat: added logic to compute failed/passed rows

* feat: add support for row level computation in schema

* chore: add test definition migration

* feat: add logic to explicitly compute row level failure

* chore: clean up code

* style: fix java

* style: fix pyton format

* fix: unhidde API for incident manager

* style: fix java styling
2023-12-27 13:38:51 +01:00
Saurabh Yadav
0691c96eaf
FIxes 13543: Added Duckdb Ingestion Connector (#14468)
* changes

* Added duck db ingestion connector
2023-12-25 18:07:09 -08:00
Shilpa Vernekar
0dfa1e2de1
Broken links corrected (#14489) 2023-12-22 13:32:47 -08:00
Pere Miquel Brull
1339d6518f
MINOR - Fix py CI (#14488) 2023-12-22 16:27:21 +01:00
Pere Miquel Brull
b786064bc2
#11857 - Store workflow status in the Ingestion Pipeline Status (#14462)
* Register StackTraceError in spec

* Register StackTraceError in spec

* Register StackTraceError in spec

* Add todos

* Update status

* docs

* format

* Fix tests

* Fix tests

* Fix tests

* Ignore generated

* Fix tests

* Fix tests

* Tests

* Try constants

* Try constants

* Print

* Print

* Print

* order

* Fix service name

* fix ui error

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-12-22 15:43:50 +01:00
Pere Miquel Brull
7e8e4a7e68
#14320 - FIx Airflow Callback datetime conversion (#14487)
* #14320 - don't cast twice

* #14320 - Fix Airflow Callback datetime conversion

* import
2023-12-22 15:43:41 +01:00