3270 Commits

Author SHA1 Message Date
Teddy
1a18c7d7f8
fix: default to raw dataset (#20192) 2025-03-11 15:52:39 -07:00
Teddy
e0a38e1146
fix: return None for BQ partition handler (#20169)
* fix: return None for BQ partition handler

* fix: change log level to info
2025-03-11 15:52:12 -07:00
Suman Maharana
9ddda9c246
Fix table constraint error (#20191) 2025-03-11 17:55:15 +00:00
harshsoni2024
aedbe8be2d
fix pbi, vertica, metabase tests (#20190) 2025-03-11 16:40:56 +01:00
Mayur Singal
d4205a5a13
Fix #20048: Ingestion Fails for Snowflake ARRAY Column (#20134) 2025-03-11 18:19:58 +05:30
Pere Miquel Brull
467534c088
Revert "Fixes 20063: Allowing to trigger an app with a payload (#20056)" (#20155)
This reverts commit 9642c7353813be7a450bba4148d5b4eb751b1c33.
2025-03-10 06:59:29 +01:00
IceS2
9642c73538
Fixes 20063: Allowing to trigger an app with a payload (#20056)
* Allowing to trigger an app with a payload

* added option to pass custom payload to triggered job

* feat(apps): add custom config to trigger

- added custom config for app triggers

* added handling for configuration validation on apps

* - removed `oneOf` from app configuration scheme
- generated ts files for apps

* added CONFIG_OVERRIDE_KEY

* format

* fixed app config key

* format

* format

* fixed python app runner to work with new schema

---------

Co-authored-by: sushi30 <imri.paran@gmail.com>
2025-03-07 21:17:05 +05:30
Suman Maharana
5d559cf317
Fix - Schema url contructed wrong (#20120) 2025-03-07 19:18:40 +05:30
Pere Miquel Brull
2e7c9a0875
FIX #19765 - Improve Column Name Scanner (#20136) 2025-03-07 14:32:59 +01:00
harshsoni2024
40a9c67875
Day 1 - Dashboard service lineage without db_service_name (#19911) 2025-03-07 11:16:58 +05:30
Suman Maharana
adc50de9f4
Fix - Manual constraints deleted (#20118) 2025-03-06 22:09:31 +05:30
Mayur Singal
c1592b54c5
Capture Query Cost (#19991) 2025-03-06 19:49:59 +05:30
Akash Verma
af9d7c9fa3
Fixes #15696: Added Powerbi filter query (#19912)
* Powerbi filter query

* error handling

* Added test and minor changes

* Checkstyle

* resolved comments

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
2025-03-06 18:24:57 +05:30
Suman Maharana
137f86c00d
Day1 - Pipeline Usage (#19960) 2025-03-06 18:19:42 +05:30
Suman Maharana
0cd35015d7
#1049 Day 1 Dashboard Usage - Tableau, Looker (#19796) 2025-03-06 18:15:28 +05:30
Suman Maharana
4d464c3173
Fix dbt Nonetype not Iterable (#20073) 2025-03-06 18:14:25 +05:30
Suman Maharana
29faee5b16
Fix Sigma not ingesting workbooks (#20082) 2025-03-05 17:21:20 +05:30
harshsoni2024
b14b4f5ac8
snowflake vector & unknown col. fix (#20079) 2025-03-05 17:00:14 +05:30
Pere Miquel Brull
55d7e50441
MINOR - Add and remove data products Actions in Automator (#19948)
* MINOR - Add and remove Data Product assets in Automator config

* MINOR - Add and remove Data Product assets in Automator config

* domain mixin

* build ref

* build ref

* create types

* fix tests

* fix conflicts

---------

Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
2025-03-05 07:11:17 +01:00
olof-nn
c46ed2aa56
Don't overwrite query to execute (#20069) 2025-03-05 11:11:20 +05:30
Mayur Singal
6f0dbd01eb
MINOR: Redshift - better log for view with no schema binding (#20005) 2025-03-04 15:26:20 +05:30
Mayur Singal
2af5e30f4c
MINOR: Improve memeory in temp table lineage (#20041) 2025-03-04 15:24:47 +05:30
harshsoni2024
de72e87bdf
mysql sample data createTable order fix (#20019) 2025-02-28 18:51:13 +05:30
Akash Verma
dfb6e1e3fe
Feat: Mssql Db&schema descrption (#19769) 2025-02-28 12:13:23 +05:30
Sriharsha Chintalapani
799e49e391
Search: improve relevancy for plural/singular words, partial matches,… (#20000)
* Search: improve relevancy for plural/singular words, partial matches, exact matches

* apply to all indexes

* Fix other query patterns

* Revert changes of database and databaseSchema fields in TableIndex.getFields() and table index mapping

* add missing boost query builder in es

* fix ci

* add max_ngram_diff setting in di-assets index

* fix TestCaseResourceTest mvn test failure

---------

Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2025-02-27 16:47:08 +01:00
Mayur Singal
8479fc436e
MINOR: Looker case insensitive search for users (#19979) 2025-02-27 14:27:04 +05:30
Mayur Singal
73876a9122
MINOR: Use clean url in clients (#19978) 2025-02-27 12:18:50 +05:30
Pere Miquel Brull
603d61eaa2
Fix #19856 - Set the db in mysql/mariadb for metric computations (#19994)
* use db for mysql/mariadb

* format
2025-02-26 20:30:42 +01:00
Pere Miquel Brull
6b7a9fe76c
MINOR - Update sampler tablenames (#19976)
* MINOR - Update sampler tablenames

* MINOR - Update sampler tablenames

* MINOR - Update sampler tablenames
2025-02-26 14:08:14 +01:00
Imri Paran
97fad806a2
Fixes 19755: Publish app config with status (#19754)
* feat(app): add config to status

add config to the reported status of the ingestion pipeline

* added separate pipeline service client call for external apps

* fix masking of pydantic model

* - overload model_dump to mask secrets instead of a separate method
- moved tests to test_custom_pydantic.py

* fix: execution time

* fix: mask secrets in dump json

* fix: for python3.8

* fix: for python3.8

* fix: use mask_secrets=False when dumping a model for create

* format

* fix: update mask_secrets=False for workflow configurations

* fix: use context directly when using model_dump_json

* fix: default behavior when dumping json

* format

* fixed tests
2025-02-25 16:51:49 +00:00
Keshav Mohta
df193bd0c4
Feature: Reverse Metadata Ingestion (#19723) 2025-02-25 18:41:08 +05:30
Pere Miquel Brull
51c86f5a76
VERSION - Bump ingestion debian to bookworm (#19917)
* VERSION - Bump ingestion debian to bookworm

* MINOR - Fix collate app yaml reader with unwanted dropwizard java props
2025-02-24 10:45:25 +01:00
Charalampos Georgiadis
3366385e15
Add result_maker check for query share url in Looker (#19833)
* Add result_maker check for query share url

* Formatting

* Move sourceUrl logic outside of the yield

* Rename

* Convert to snake case

* Fix camel case

* Avoid nested conditionals

* Update result_maker check

---------

Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
2025-02-20 21:46:18 +05:30
Pere Miquel Brull
2b32808011
MINOR - Upgrade Airflow to 2.10.5 (#19840)
* MINOR - Bump Ingestion versions

* MINOR - Airflow bump

* MINOR - Set Airflow 2.10.5
2025-02-20 17:11:38 +01:00
Mayur Singal
88f615ae47
Fix #19876: Snowflake Lineage Ingestion Fails Due to KeyError (#19878) 2025-02-20 16:26:11 +05:30
Sriharsha Chintalapani
a924064c09
Fix #17723: Generate Incremental Change Events even when consolidation of events applied (#19550)
* Fix #17723: Generate Incremental Change Events even when consolidation of events applied

* Fix #17723: Generate Incremental Change Events even when consolidation of events applied

* fix tests

* Fix tests

* clean policy tests

* update search methods to use incrementalChangeDescription part-1

* Fix the version page playwrights

* update search methods to use incrementalChangeDescription part-2

* introduce new field incrementalChangeDescription for search part-3

* fix mvn endpoint test

* fix followers and page search test

* fix following of assets

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
2025-02-20 10:23:08 +05:30
Mayur Singal
6305383a80
Minor: Trino Add missing import (#19853)
* Minor: Add missing import

* format
2025-02-18 10:02:31 +00:00
Pere Miquel Brull
69c9102da1
MINOR - Bump Ingestion versions (#19836)
* MINOR - Bump Ingestion versions

* MINOR - Bump Ingestion versions

* fix

* fix db_scheme for airflow +2.9.1

* fix
2025-02-18 07:56:46 +01:00
harshsoni2024
e0237d68b8
MINOR: powerbi fetch workspaces failure handle (#19785) 2025-02-17 15:27:03 +05:30
tarunpandey23
8d16657a15
fix: updated Dockerfile to fix the workflow stuck issue (#19828)
* added for test

* added for test

* Updated Dockerfile to fix the workflow stuck issue

* Updated and fixed the workflow stuck issue

* Updated and fixed the workflow stuck issue

---------

Co-authored-by: Tarun <tarun.p@deuexsolutions.com>
2025-02-17 14:57:24 +05:30
Pere Miquel Brull
91b62fdc32
FIX #19798 - Shortening SQA __tablename__ to avoid hitting errors in … (#19809)
* FIX #19798 - Shortening SQA __tablename__ to avoid hitting errors in postgres

* fix tests

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2025-02-17 09:37:06 +01:00
Pere Miquel Brull
e4c34210ec
Fix #19807 - Implement the right SQA Sampler for UnityCatalog (#19811) 2025-02-16 11:58:27 -08:00
Suman Maharana
d6f2c9c416
Fix dbt Test case Timestamp issue (#19803) 2025-02-15 20:33:02 +05:30
Akash Verma
c6fe281de0
Mssql stored procedure description (#19775)
* Mssql stored procedure description

* resolved comment

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
2025-02-14 10:47:13 +05:30
Mayur Singal
c8e24e5f3b
MINOR: Fix unity catalog lineage - handle errors (#19791)
* MINOR: Fix unity catalog lineage - hanle errors

* lint
2025-02-14 10:45:31 +05:30
Mayur Singal
365fe7daa0
Minor: Fix mysql & snowflake lineage e2e (#19767) 2025-02-13 08:44:01 +05:30
mgorsk1
0e260ce36a
feat: use dataset sql for superset lineage (#19617)
* 🎉 Init

* add fixes around row data type and backport fixes around array type

* refactor

* narrow down column lineage

* narrow down column lineage

* fix insert into

* review remarks + linter

* fix tests

* fix style check

* bring back compatibility with older python versions

* add unit tests

* fix tests

* linting

* expand tests

* make compatible with python 3.8

* checkstyle fix

* fix schema alias

* refactor - move get_dashboard_data_model_column_fqn to superset.utils as it's superset specific and won't work with other dashboard systems integrations
2025-02-12 07:14:31 -08:00
Teddy
03de0ed549
MINOR: Added missing test dependencies (#19756)
* fix: added missing test dependencies

* style: ran python linting
2025-02-12 07:01:41 -08:00
Mayur Singal
e873ba0f1b
Fix #19201: Fix connection password masking (#19749) 2025-02-12 16:16:25 +05:30
Mayur Singal
45efd878cb
Fix #19634: use system.query.history table for lineage - databricks (#19744) 2025-02-12 13:29:07 +05:30