2362 Commits

Author SHA1 Message Date
Mayur Singal
4e02cb1c54
Fix #14764: Profiler make compute metric configurable (#14791) 2024-01-19 18:13:26 +05:30
Onkar Ravgan
f10b5a3617
Fixed athena e2e db (#14774) 2024-01-19 11:51:38 +05:30
Teddy
9c4ec6c760
fix: added fall back to table metric computation (#14771) 2024-01-19 00:33:57 +05:30
Ayush Shah
831fce5b7e
Fixes 10709: Add useFqnForFiltering to profiler workflow (#14717) 2024-01-18 18:52:43 +05:30
Onkar Ravgan
c5c7171036
MINOR: Added Support for Bigquery and Redshift lifeCycle data (#14622)
* Added BQ and redshift lifeCycle

* added usage fixes
2024-01-18 15:36:43 +05:30
Ayush Shah
0d0d820305
MINOR: Add Static Sample Data, Modify Lineage (#14752) 2024-01-17 20:07:49 +05:30
NiharDoshi99
6620389ad1
fix null prop. when entire col is null (#14758) 2024-01-17 16:00:06 +05:30
Teddy
c90a86b8ad
chore: remove typing-extension dependency (#14757) 2024-01-17 09:58:10 +00:00
NiharDoshi99
c120ae98a4
#14727 change all all_ query to dba_ query (#14748) 2024-01-17 12:38:42 +05:30
Pere Miquel Brull
eadda0e3f1
#9269 - Add MSSQL Stored Procedure Support (#14739)
* Remove unnecessary field

* Remove unnecessary field

* Support query comments in MSSQL

* Remove unnecessary field

* Format

* Add external type

* Add MSSQL SP support
2024-01-17 06:37:27 +01:00
Mayur Singal
dccfd9a776
Minor: Add Column Support in Protobuf Parser (#14745) 2024-01-17 09:31:27 +05:30
Onkar Ravgan
13b326d11c
Fix 14040: Part 3 Added updated field to status for patched entities (#14708)
* Added updated field to status

* Fixed e2e test
2024-01-16 17:53:05 +05:30
Onkar Ravgan
64a4e1afce
Fix 12180, 14158: Added LF tags to Athena (#14718)
* Added LF tags to athena

* fixed pytests

* Added docs
2024-01-16 14:24:31 +05:30
NiharDoshi99
54d34934c1
#14630 added oracle stored procedures (#14641) 2024-01-15 18:28:27 +05:30
Ayush Shah
709fa7415f
Fixes 13343: Fix Superset table name issue (#14714) 2024-01-15 18:19:27 +05:30
Ayush Shah
98eae7154a
Fix Airflow Source Url (#14712) 2024-01-15 12:31:43 +00:00
Mayur Singal
355f933ef0
Minor: Fix Superset API Test Connection (#14696) 2024-01-15 15:25:48 +05:30
Mayur Singal
21a701f360
Minor: Fix Postgres Cypress Failure (#14698) 2024-01-13 11:49:36 +05:30
Onkar Ravgan
f1049d55c4
Fixed mode test connection (#14693) 2024-01-12 18:29:12 +05:30
Mayur Singal
a7c17c5351
Fix #14653: Fix Unity Catalog no children struct issue (#14688) 2024-01-12 12:59:50 +05:30
Shiyang Xiao
241f3c68d7
Fixes #14413: Add SAS connector (#14415)
* feat: SAS Viya connector

* refactor SASCatalog to SAS

* add SAS logo to UI and connection documentation

* doc changes

* modify ingestion logic

* revert original changes

* added support for dataflow & perfect logic for reports/datatables

* add filter doc

* more updates to perfect ingestion for each asset type

* fix a bug with table lineage not created properly

* Delete ingestion/pipelines/sasCatalog.yaml

* precomit fix

* Conversion to database connector

* minor fixes

* make custom properties type generic

* Add SAS javaEnum

* add dummy variable for sas.yaml

---------

Co-authored-by: lizmc <liz.mcintosh@sas.com>
Co-authored-by: Shiyang Xiao <Shiyang.Xiao@sas.com>
2024-01-11 06:46:57 -08:00
Mayur Singal
66d92300c6
Fix #13894: Ingest only open search indexes ES (#14685) 2024-01-11 18:23:49 +05:30
Ayush Shah
9c6d202555
Add Sample data, modify regex pattern (#14467) 2024-01-11 14:23:33 +05:30
Onkar Ravgan
aee42cf72b
removed patch warnings (#14669) 2024-01-10 23:02:30 +05:30
Mayur Singal
810e1df051
Minor: Fix Databricks Pipeline Test Connection Fix (#14667) 2024-01-10 18:37:00 +05:30
Mayur Singal
8197fe56ec
Minor: Snowflake - Add Support For Materialised Views (#14664) 2024-01-10 18:36:42 +05:30
Onkar Ravgan
d913586622
Fixed auth header in rest client (#14632) 2024-01-09 11:47:09 +05:30
Pere Miquel Brull
24643a397a
#14492 - Fix Snowflake SP parsing with empty signature (#14623) 2024-01-08 11:16:35 -08:00
Pere Miquel Brull
e8eb2bda3c
MINOR - Hide AutoTagger, add MetaPilot search, and fix apps status calls (#14201)
* MINOR - Hide AutoTagger & MetaPilot search & Fix apps status calls

* Fix method call

* Fix method call

* Fix method call

* "additionalProperties": true

* Use EntityRef

* Use ES sink

* Update UI schema type
2024-01-08 14:57:07 +01:00
Pere Miquel Brull
b92fd5cde4
MINOR - Move test case incident to the top of the data_quality_data_time_series (#14600)
* Add field and index

* MINOR - Move test case incident to the top of the data_quality_data_time_series table

* Fix test

* Fix compile

* Format

* Update incidentId

* Rename field

* Fix patch
2024-01-08 07:42:15 +01:00
Onkar Ravgan
ecdb7b9f41
Fixes 14109 and 14325: Optimised Tableau Connector (#14548)
* Optimised tableau conn

* Added comment
2024-01-08 06:33:05 +01:00
Ayush Shah
d513dc4d4d
#11890: Add GCS Json schema changes (#14559)
* Issue-11890: Add GCS Json schema changes

* Modify Gcs Security config

* Add Test Def for gcs

* Add gcs for storage_metadata_config
2024-01-06 19:27:36 -08:00
Mayur Singal
a789fc86d6
Fix #13053: Remove Connection URI config MongoDB (#14584)
* Fix #13053: Remove Connection URI cofig MongoDB

* pyformat & test fixes
2024-01-05 10:51:12 -08:00
Mayur Singal
d82770a91e
Fix #14421: Airflow - Handle pipeline status when dag killed (#14562) 2024-01-05 18:14:26 +05:30
Pere Miquel Brull
0255171218
MINOR - Create Test Case Resolution ts entry & delete resolution when… (#14541)
* MINOR - Create Test Case Resolution ts entry & delete resolution when Test Case is deleted
2024-01-05 09:15:49 +01:00
Mayur Singal
bf74ef9e22
Minor: Filter Create SP & Declare Queries for MSSQL Lineage (#14586) 2024-01-05 12:13:54 +05:30
Onkar Ravgan
7fdff15648
fixed dbt runresults ts (#14572) 2024-01-04 12:36:23 +05:30
Onkar Ravgan
a892267fbb
Fixed salesforce table req (#14570) 2024-01-04 10:45:58 +05:30
Mayur Singal
d197164125
Fix #14176: Add AzureSQL dialect in lineage (#14563) 2024-01-03 08:11:05 -08:00
NiharDoshi99
78bd8f98b5
#10203 delete temp creds files (#14536) 2024-01-02 13:09:15 +05:30
Cristian Calugaru
1631172528
Issue 10587 - Storage Service doesn't handle deleted containers (#14497)
* global manifest option for storage services

* added a no metadata config source option for global manifest s3 services option

* merge fixes

* more merge fixes.

* black stuff

* test fixes

* formatting

* storage service container automatic deletion support

* reverted auto-modified files.

* styling

* linting
2023-12-30 13:27:16 -08:00
mgorsk1
5ae83b284a
Fixes 14528: skip statuses for tasks that are no longer present in Airflow DAG (#14529)
* 🐛 Fix attribute name

* import AirflowTask model

* make py_format

* make py_format_check
2023-12-30 13:26:34 -08:00
Pere Miquel Brull
f4bbca3f72
MINOR - Clean topology & add tests (#14527)
* Clean topo

* Format

* Add tests

* Fix tests

* Merge main
2023-12-29 17:00:59 +01:00
Pere Miquel Brull
3613d0b18a
MINOR - Airflow Lineage Backend to verify the fingerprint (#14530)
* hash

* MINOR - Airflow Lineage Backend to verify the fingerprint
2023-12-29 16:36:28 +01:00
Takahiro Nakayama
7d05d9f50a
Fixes: GetTags-403 Permission denied on resource project None. (#14521)
* Fix "GetTags-403 Permission denied on resource project None."

Signed-off-by: Takahiro Nakayama <civitaspo@gmail.com>
2023-12-29 15:09:59 +05:30
Pere Miquel Brull
1c2219517e
#13649 - Add Pipeline Status Skipped (#14524)
* #13649 - Add Pipeline Status Skipped

* Format
2023-12-29 09:41:59 +01:00
Pere Miquel Brull
4b6034c527
#13638 - Filter task instance by removed state (#14515)
* Clean name logging and filter out removed tasks

* Update client logging

* Update client logging
2023-12-29 06:56:44 +01:00
Pere Miquel Brull
b84ce33b80
#11799 - Fix Airfow ownership & add pipeline tasks (#14510)
* Fix airflow owner and add tasks

* Add pipeline tasks ownership

* MINOR - Fix py CI

* Add pipeline tasks ownership

* Add pipeline tasks ownership

* MINOR - Fix py CI

* MINOR - Fix py CI

* Add pipeline tasks ownership

* patch team

* patch team

* Format
2023-12-28 10:25:00 -08:00
Mayur Singal
0c14a73fd1
Minor: Handle Column Lineage Exceptions (#14505)
* Minor: Handle Column Lineage Exceptions

* py format

* pylint
2023-12-27 13:04:10 -08:00
Teddy
61ef55290e
MINOR - generic profiler optimization for sampling and BQ (#14507)
* fix: limit sampling to specific column

* fix: handle bigquery struct columns

* fix: default partition to 1 DAY for BQ

* fix: default to __TABLES__ for BQ table metrics

* style: ran python linting

* style: fix linting

* fix: python style

* fix: set partition to DAY if not HOUR
2023-12-27 19:13:44 +01:00