2636 Commits

Author SHA1 Message Date
IceS2
e3c3f2ac33
Fix E2E count (#16654) 2024-06-14 08:21:07 +00:00
Trs
fc9033b953
Fixes(ingestion/source/dbt): Handle None Type in get_tag_labels Function for DBT Metadata Processing (#16648)
* fix condition

* fix

* lint
2024-06-13 17:19:46 +05:30
Ayush Shah
90d9ef639e
Fix Arg for qliksense (#16626) 2024-06-13 09:01:45 +05:30
IceS2
328ed2bf11
MINOR: Fix e2e (#16627)
* Fix Metabase E2E Test

* Add 'debug' input to python e2e tests

* Fix 'debug' default to be 'false'

* Standardized all Metabase IDs to MetabaseStrId

* Fix Metabase expected filtered sink mix value

* Fix wrong parameter being passed to the config

* Fix powerBI e2e tests

* Fix one Redash e2e test

* Fix checkstyle

* Fix Dashboard create patch_request not using EntityReferenceList

* Fix Redash E2E test value

* Add logging to create patch request

* Fix checkstyle and linting

* Fix default debug value

* Fix e2e workflow

* Fix e2e workflow

* Fix e2e workflow

* Fix metabase and powerbi e2e values
2024-06-12 19:32:45 +02:00
Mayur Singal
b7ef13bc95
MINOR: Fix ssl connection in usage & lineage (#16625) 2024-06-12 16:29:24 +05:30
Ayush Shah
2f8de77140
Add Test Suite SSL (#16619) 2024-06-12 14:00:24 +05:30
Mayur Singal
7359d6210c
MINOR: Fix Profiler for SSL Enabled Source (#16613) 2024-06-12 11:40:30 +05:30
Onkar Ravgan
4f50e0b6a0
cleaned dbt code after pydantic update (#16605) 2024-06-12 11:24:29 +05:30
Matt Chamberlin
5236950b6d
Fixes 16595: Parse Iceberg REST table FQN from identifier (#16596) 2024-06-11 15:07:57 +05:30
IceS2
070bd2570e
MINOR: Fix e2e (#16594)
* Fix Metabase E2E Test

* Add 'debug' input to python e2e tests

* Fix 'debug' default to be 'false'

* Standardized all Metabase IDs to MetabaseStrId

* Fix Metabase expected filtered sink mix value

* Fix wrong parameter being passed to the config

* Fix powerBI e2e tests

* Fix one Redash e2e test

* Fix checkstyle
2024-06-11 13:19:25 +05:30
Teddy
ff00175fb1
MINOR - Data Quality Preparatory Work (#16523)
* chore: added min/max bound and dynamic support flag

* chore: added migration for dynamic attribute

* style: fix pythin checkstyle
2024-06-10 11:03:55 +02:00
Pere Miquel Brull
d2bba3b8e0
FIX - e2e test for pydantic v2 (#16577)
* FIX - e2e test for pydantic v2

* FIX - e2e test for pydantic v2

* FIX - e2e test for pydantic v2
2024-06-10 08:04:36 +02:00
Pere Miquel Brull
93f976f6e4
FIX - e2e test for pydantic v2 (#16576)
* FIX - e2e test for pydantic v2

* FIX - e2e test for pydantic v2

* FIX - e2e test for pydantic v2
2024-06-08 10:07:58 +00:00
Pere Miquel Brull
f5996b2f3a
FIX - e2e test for pydantic v2 (#16572) 2024-06-07 22:52:16 +02:00
Onkar Ravgan
38e2793705
MINOR: Enabled pbit file test (#16531)
* Enabled pbit file test

* Updated test

* update test

* fixed pylint
2024-06-07 17:59:45 +05:30
Pere Miquel Brull
cb72a22b59
Fix - e2e tests for pydantic V2 (#16551)
* Fix - e2e tests for pydantic V2

* add correct default

* add correct default

* revert datetime aware

* revert datetime aware

* revert datetime aware

* revert datetime aware

* revert datetime aware

* revert datetime aware

* revert datetime aware

* revert datetime aware

* fix apis

* format
2024-06-06 19:36:17 -07:00
Mohit Yadav
9ec3d94e3b
[FIX] GlossaryTerm reviewers should be user or team only (#16372)
* add teams as reviewer

* Check Users to be reviewers

* Reviewers can be a team or user

* Fix check by id or name

* Review can be team or user both

* Validate Reviewers

* add multi select control

* - Fix Reviewers

* - Centralize Reviewer Relationship to EntityRepository

* - Sort

* add team as reviewer for glossary terms

* locales

* cleanup

* - Update Reviewer should remove existing reviewers

* fix selectable owner control

* fix code smells

* fix reviewer issue

* add glossary cypress

* fix patch issue on reviewers set to null

* update cypress tests

* fix cypress

* fix cypress

* fix reviewers in glossary task and supported cypress

* fix pytest

* Fix

* fix cypress

* fix code smells

* Inherited Reviewers need to be present always

* filter out inherited users

* fix cypress

* fix backend tests failure

* fix backend tests failure -checkstyle

* restrict owner to accept task in case of reviewer present

* fix pytest

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
2024-06-06 20:23:37 +05:30
Pere Miquel Brull
d8e2187980
#15243 - Pydantic V2 & Airflow 2.9 (#16480)
* pydantic v2

* pydanticv2

* fix parser

* fix annotated

* fix model dumping

* mysql ingestion

* clean root models

* clean root models

* bump airflow

* bump airflow

* bump airflow

* optionals

* optionals

* optionals

* jdk

* airflow migrate

* fab provider

* fab provider

* fab provider

* some more fixes

* fixing tests and imports

* model_dump and model_validate

* model_dump and model_validate

* model_dump and model_validate

* union

* pylint

* pylint

* integration tests

* fix CostAnalysisReportData

* integration tests

* tests

* missing defaults

* missing defaults
2024-06-05 21:18:37 +02:00
Onkar Ravgan
e9ec4367ee
Fixed quicksight conn (#16537) 2024-06-05 08:54:41 +00:00
Onkar Ravgan
0106963e7e
fixed tableau e2e (#16534) 2024-06-05 14:21:35 +05:30
Suman Maharana
cdca199ec8
Fix #16370 - Added view/DDL definition rule (#16486)
* added rule includeDDL

* code optimize

* review changes
2024-06-05 11:11:04 +05:30
Onkar Ravgan
29028eabb3
fixed oracle e2e count (#16530) 2024-06-04 20:37:21 +05:30
Teddy
656da03b14
fix: None type is not iterable (#16496) 2024-05-31 10:17:28 +00:00
Mayur Singal
22be0edd79
MINOR: Ignore Cluster Information from columns (#16495) 2024-05-31 15:41:12 +05:30
Ayush Shah
a98f6b8dd0
MINOR: Kafka Setup SSL Arg Fix (#16469) 2024-05-30 16:03:03 +05:30
Suman Maharana
5ba7769b76
Fix #16425 : Correct Schema Definition for Databricks (#16441)
* Fix Incorrect Schema Definition for Databricks

* Catalog name is showing correctly
2024-05-30 12:29:12 +05:30
Suman Maharana
0677c4794f
Only Parse view query (#16470) 2024-05-30 11:55:33 +05:30
Imri Paran
c8dd9971b4
added usage query to logs in case of error (#16455) 2024-05-29 17:14:03 +00:00
Antoine Balliet
0abd3ca5fe
fix: ingestion for dbt > 1.8.0 resource_type is not an enum (#16415)
* fix: resource_type is not an enum

* feat: add log to display finis

* improve readability

* use getattr to be compatible

* format
2024-05-28 17:00:04 +02:00
Imri Paran
067fb510ab
MINOR: test case for usage->delete->usage (#16409)
* test: added test case to demonstrate cache issue
2024-05-28 11:23:43 +02:00
Imri Paran
a4c516d2c7
Fixes 16305: Added Test Case for Matching Enum (#16362)
* Added Test Case for Matching Enum

1. Implemented the test case using the `matchEnum` parameter.
2. Added integration tests.
3. Added migrations.

* fix tests

* fixed tests

* format

* fixed tests

* clear search cache before running ingestion

* format

* changed scopt of aws fixture

* moved migrations to 1.5.0
2024-05-28 09:30:30 +02:00
gpby
d909a3141e
Teradata Connector (#16373)
* [WIP] add teradata connector

* [WIP] add teradata ingestion

* [WIP] add teradata connector

* [WIP] add teradata connector

* [WIP] add teradata connector

* [WIP] add teradata connector

* [WIP] add teradata connector

* [WIP] add teradata connector

* Reformat code

* Remove unused databaseName property
2024-05-28 06:40:22 +02:00
Maxim Martynov
18b2a22958
Fix OpenLineage ingestor (#16416)
* Fix OpenLineage ingestor

* py format

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2024-05-27 13:47:01 +05:30
Ayush Shah
e1a4937744
Fix SSL issue (#16412) 2024-05-26 18:49:35 +00:00
juntao
8dd613caa5
Fixes #16235: need quote fullyQualifiedName in Ingestion Framework (#16273)
* Fixes #16235: need quote fullyQualifiedName in Ingestion Framework

* MINOR: fix UT issue

* revert: fix UT issue

* revert code

* revert code

* format code
2024-05-23 17:45:47 +02:00
Akash Jain
015d71c9de
chore: Update main branch for 1.5.0 Release (#16401) 2024-05-23 18:46:15 +05:30
Pere Miquel Brull
17aed8a9e9
MINOR - Fix GX version (#16394) 2024-05-22 19:25:42 +00:00
Imri Paran
d5bf30ccd3
MINOR: trino integration test (#16291)
* added trino integration test

* - removed warnings for classes which are not real tests
- removed "helpers" as its being used

* use a docker network instead of host

* print logs for hive failure

* removed superset unit tests

* try pinning requests for test

* try pinning requests for test

* wait for hive to be ready

* fix trino fixture

* - reduced testcontainers_config.max_tries to 5
- remove intermediate containers

* print with logs

* disable capture logging

* updated db host

* removed debug stuff

* removed debug stuff

* removed version pin for requests

* reverted superset

* ignore trino integration on python 3.8
2024-05-22 15:12:00 +00:00
Mayur Singal
8845823758
MINOR: remove newline char in lineage queries (#16361) 2024-05-22 16:35:38 +05:30
Mayur Singal
a2c68f6901
MINOR: Fix UC missing table type (#16376) 2024-05-22 14:13:01 +05:30
Mayur Singal
89829949ce
MINOR: Fix flaky pytests (#16379) 2024-05-22 14:12:34 +05:30
harshsoni2024
a1a68ae73b
restrict requests version on setup (#16365) 2024-05-21 18:13:37 +05:30
Mayur Singal
ed792fd77b
MINOR: DDL Change log level to debug (#16360) 2024-05-21 15:13:55 +05:30
Teddy
4de6fae26e
Minor: None query res check on table metrics (#16334)
* fix: None query res check on table metrics

* style: ran python linting
2024-05-20 10:52:21 -07:00
Mayur Singal
199613fab0
MINOR: Do not populate schema def if not found (#16345) 2024-05-20 20:06:33 +05:30
Suman Maharana
bd3f47a563
MINOR - Added quicksight pydantic models (#16269)
* Added quicksight pydantic models

* pyformat

* resolved type hints

* Renamed sheet -> chart in models
2024-05-17 08:40:20 +02:00
Mayur Singal
1798b647c3
MINOR: Bump Collate Sqllineage Version (#16293) 2024-05-17 08:39:37 +02:00
Mayur Singal
7b52f5a6b0
MINOR: Fix kafka connection storage (#16313) 2024-05-17 11:59:08 +05:30
Pere Miquel Brull
263afbeb5c
MINOR - pkg_resources is deprecated (#16316) 2024-05-17 07:56:07 +02:00
harshsoni2024
d6046c811c
MINOR: superset pytest fix (#16299)
* superset testcontainer image tag fix

* update chart data acc. to runtime
2024-05-17 07:27:36 +02:00