2958 Commits

Author SHA1 Message Date
Akash Verma
cf7a442e32
Fixes #19891 : Added measures in powerbi (#19990) 2025-03-17 14:43:22 +05:30
Teddy
fbea55e5d3
feat: allow all as an argument in include columns (#20263)
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
2025-03-17 09:10:12 +01:00
Mayur Singal
d30fd90096
Minor: Query Cost Table Aggregation Endpoint (#20270) 2025-03-17 11:33:50 +05:30
Keshav Mohta
b21c60304c
Fixes #19688 - Added Postgres Functions Ingestion (#20188) 2025-03-15 11:54:38 +05:30
Mayur Singal
b727f76ce4
MINOR: Add source url in sample data (#20133) 2025-03-14 11:42:51 +05:30
Katarzyna Kałek
397dd0512f
Fixes #19619 (#19620)
* fixed s3 access bug for parquet files

* fixed formatting

* parsed endpoint_override to str in s3 parquet ingestion

---------

Co-authored-by: Katarzyna Kałek <kkalek@olx.pl>
2025-03-14 11:26:23 +05:30
Katarzyna Kałek
168e29423a
Fixed iteration over None in Tableau ingestion (#20246)
Co-authored-by: Katarzyna Kałek <kkalek@olx.pl>
2025-03-14 09:37:09 +05:30
Mayur Singal
f291e5cb1d
MINOR: Add cost in sample usage queries (#20211) 2025-03-13 19:27:03 +01:00
Mayur Singal
c5c724887e
Capture BQ Query Cost (#20124) 2025-03-13 17:06:35 +05:30
Suman Maharana
a964cda244
Fix yield tag error (#20221) 2025-03-13 10:43:56 +05:30
Mayur Singal
faa768fc16
MINOR: Fix incremental lineage if processedLineage is null (#20219) 2025-03-13 10:32:51 +05:30
Ayush Shah
b9540b0c7d
Minor: Fix Superset Dashboard Lineage Handling (#20207)
* Fix Superset Dashboard Lineage Handling

- Safely handle chart SQL attribute using getattr
- Update error message for dashboard lineage details to be more generic

* refactor: formatting improvement in Superset dashboard lineage handling
2025-03-12 23:42:10 +05:30
Mayur Singal
a061367f97
Incremental Lineage Processing (#20185) 2025-03-12 21:21:28 +05:30
Keshav Mohta
8a6ab74265
Fixes #20007 - DBT Cloud Ingestion Fail (#20162) 2025-03-12 17:24:52 +05:30
harshsoni2024
826279608f
issue-19892: parse powerbi table source (#20141) 2025-03-12 12:59:29 +05:30
Teddy
1a18c7d7f8
fix: default to raw dataset (#20192) 2025-03-11 15:52:39 -07:00
Teddy
e0a38e1146
fix: return None for BQ partition handler (#20169)
* fix: return None for BQ partition handler

* fix: change log level to info
2025-03-11 15:52:12 -07:00
Suman Maharana
9ddda9c246
Fix table constraint error (#20191) 2025-03-11 17:55:15 +00:00
Mayur Singal
d4205a5a13
Fix #20048: Ingestion Fails for Snowflake ARRAY Column (#20134) 2025-03-11 18:19:58 +05:30
Pere Miquel Brull
467534c088
Revert "Fixes 20063: Allowing to trigger an app with a payload (#20056)" (#20155)
This reverts commit 9642c7353813be7a450bba4148d5b4eb751b1c33.
2025-03-10 06:59:29 +01:00
IceS2
9642c73538
Fixes 20063: Allowing to trigger an app with a payload (#20056)
* Allowing to trigger an app with a payload

* added option to pass custom payload to triggered job

* feat(apps): add custom config to trigger

- added custom config for app triggers

* added handling for configuration validation on apps

* - removed `oneOf` from app configuration scheme
- generated ts files for apps

* added CONFIG_OVERRIDE_KEY

* format

* fixed app config key

* format

* format

* fixed python app runner to work with new schema

---------

Co-authored-by: sushi30 <imri.paran@gmail.com>
2025-03-07 21:17:05 +05:30
Suman Maharana
5d559cf317
Fix - Schema url contructed wrong (#20120) 2025-03-07 19:18:40 +05:30
Pere Miquel Brull
2e7c9a0875
FIX #19765 - Improve Column Name Scanner (#20136) 2025-03-07 14:32:59 +01:00
harshsoni2024
40a9c67875
Day 1 - Dashboard service lineage without db_service_name (#19911) 2025-03-07 11:16:58 +05:30
Suman Maharana
adc50de9f4
Fix - Manual constraints deleted (#20118) 2025-03-06 22:09:31 +05:30
Mayur Singal
c1592b54c5
Capture Query Cost (#19991) 2025-03-06 19:49:59 +05:30
Akash Verma
af9d7c9fa3
Fixes #15696: Added Powerbi filter query (#19912)
* Powerbi filter query

* error handling

* Added test and minor changes

* Checkstyle

* resolved comments

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
2025-03-06 18:24:57 +05:30
Suman Maharana
137f86c00d
Day1 - Pipeline Usage (#19960) 2025-03-06 18:19:42 +05:30
Suman Maharana
0cd35015d7
#1049 Day 1 Dashboard Usage - Tableau, Looker (#19796) 2025-03-06 18:15:28 +05:30
Suman Maharana
4d464c3173
Fix dbt Nonetype not Iterable (#20073) 2025-03-06 18:14:25 +05:30
Suman Maharana
29faee5b16
Fix Sigma not ingesting workbooks (#20082) 2025-03-05 17:21:20 +05:30
harshsoni2024
b14b4f5ac8
snowflake vector & unknown col. fix (#20079) 2025-03-05 17:00:14 +05:30
Pere Miquel Brull
55d7e50441
MINOR - Add and remove data products Actions in Automator (#19948)
* MINOR - Add and remove Data Product assets in Automator config

* MINOR - Add and remove Data Product assets in Automator config

* domain mixin

* build ref

* build ref

* create types

* fix tests

* fix conflicts

---------

Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
2025-03-05 07:11:17 +01:00
olof-nn
c46ed2aa56
Don't overwrite query to execute (#20069) 2025-03-05 11:11:20 +05:30
Mayur Singal
6f0dbd01eb
MINOR: Redshift - better log for view with no schema binding (#20005) 2025-03-04 15:26:20 +05:30
Mayur Singal
2af5e30f4c
MINOR: Improve memeory in temp table lineage (#20041) 2025-03-04 15:24:47 +05:30
Akash Verma
dfb6e1e3fe
Feat: Mssql Db&schema descrption (#19769) 2025-02-28 12:13:23 +05:30
Mayur Singal
8479fc436e
MINOR: Looker case insensitive search for users (#19979) 2025-02-27 14:27:04 +05:30
Mayur Singal
73876a9122
MINOR: Use clean url in clients (#19978) 2025-02-27 12:18:50 +05:30
Pere Miquel Brull
603d61eaa2
Fix #19856 - Set the db in mysql/mariadb for metric computations (#19994)
* use db for mysql/mariadb

* format
2025-02-26 20:30:42 +01:00
Pere Miquel Brull
6b7a9fe76c
MINOR - Update sampler tablenames (#19976)
* MINOR - Update sampler tablenames

* MINOR - Update sampler tablenames

* MINOR - Update sampler tablenames
2025-02-26 14:08:14 +01:00
Imri Paran
97fad806a2
Fixes 19755: Publish app config with status (#19754)
* feat(app): add config to status

add config to the reported status of the ingestion pipeline

* added separate pipeline service client call for external apps

* fix masking of pydantic model

* - overload model_dump to mask secrets instead of a separate method
- moved tests to test_custom_pydantic.py

* fix: execution time

* fix: mask secrets in dump json

* fix: for python3.8

* fix: for python3.8

* fix: use mask_secrets=False when dumping a model for create

* format

* fix: update mask_secrets=False for workflow configurations

* fix: use context directly when using model_dump_json

* fix: default behavior when dumping json

* format

* fixed tests
2025-02-25 16:51:49 +00:00
Keshav Mohta
df193bd0c4
Feature: Reverse Metadata Ingestion (#19723) 2025-02-25 18:41:08 +05:30
Charalampos Georgiadis
3366385e15
Add result_maker check for query share url in Looker (#19833)
* Add result_maker check for query share url

* Formatting

* Move sourceUrl logic outside of the yield

* Rename

* Convert to snake case

* Fix camel case

* Avoid nested conditionals

* Update result_maker check

---------

Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
2025-02-20 21:46:18 +05:30
Mayur Singal
88f615ae47
Fix #19876: Snowflake Lineage Ingestion Fails Due to KeyError (#19878) 2025-02-20 16:26:11 +05:30
Mayur Singal
6305383a80
Minor: Trino Add missing import (#19853)
* Minor: Add missing import

* format
2025-02-18 10:02:31 +00:00
harshsoni2024
e0237d68b8
MINOR: powerbi fetch workspaces failure handle (#19785) 2025-02-17 15:27:03 +05:30
Pere Miquel Brull
91b62fdc32
FIX #19798 - Shortening SQA __tablename__ to avoid hitting errors in … (#19809)
* FIX #19798 - Shortening SQA __tablename__ to avoid hitting errors in postgres

* fix tests

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2025-02-17 09:37:06 +01:00
Pere Miquel Brull
e4c34210ec
Fix #19807 - Implement the right SQA Sampler for UnityCatalog (#19811) 2025-02-16 11:58:27 -08:00
Suman Maharana
d6f2c9c416
Fix dbt Test case Timestamp issue (#19803) 2025-02-15 20:33:02 +05:30