927 Commits

Author SHA1 Message Date
Sergio Gómez Villamor
69981675a5
feat(mssql): adds subtypes aspect for dataflow and datajobs (#12775) 2025-03-05 17:11:04 +01:00
Sergio Gómez Villamor
85d3a9d31d
feat(okta): custom properties for okta user (#12773) 2025-03-05 15:53:01 +01:00
Peter Wang
9e7f48278a
feat(ingestion/superset): ownership info for charts, dashboards and datasets (#12750) 2025-03-04 12:28:24 -08:00
skrydal
3e1b20c727
feat(ingestion/iceberg): Several improvements to iceberg connector (#12744) 2025-03-01 01:30:07 +01:00
Peter Wang
fc26c1cfb5
feat(ingestion/superset): superset dataset lineage for metadata ingestion (#12679)
Co-authored-by: Shuixi Li <llance@users.noreply.github.com>
Co-authored-by: Gabe Lyons <gabe.lyons@acryl.io>
2025-02-28 07:29:25 -08:00
Aseem Bansal
9c3bd34995
dev: enable ruff rule (#12749) 2025-02-28 17:49:52 +05:30
Aseem Bansal
0fc074b50a
dev: enable ruff rule (#12742) 2025-02-28 10:59:06 +05:30
Aseem Bansal
2101a50e83
dev: immutable args remove (#12735) 2025-02-27 15:38:47 +05:30
Rasnar
5f5e395c90
Fix platform instance support on Druid ingestion (#12716)
Co-authored-by: rasnar <11248833+Rasnar@users.noreply.github.com>
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-02-26 08:36:06 +01:00
Sergio Gómez Villamor
c980cbd8f2
tests(druid): integration tests for druid ingestion (#12717)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2025-02-25 08:08:45 +01:00
Harshal Sheth
8dfd8fbbaf
feat(ingest): migrate Cassandra source to new SDK (#12695) 2025-02-24 11:57:51 -08:00
Mayuri Nehate
fbed37b374
fix(ingest/snowflake): update logic to identify hybrid tables (#12672) 2025-02-20 11:23:57 +05:30
mihai103
dcf1576c30
fix(tableau): skip ingesting filtered project assets (#12591)
Co-authored-by: Mihai Ciocirdel <mihai.ciocirdel@swisscom.com>
Co-authored-by: Jay <159848059+jayacryl@users.noreply.github.com>
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-02-14 09:25:37 +01:00
Sergio Gómez Villamor
b20211536c
feat(okta): adds ingest_groups_users config parameter (#12371)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2025-02-13 12:00:07 +01:00
Mayuri Nehate
e2a98d217e
fix(ingest): pick topics from config for sink connector (#12535) 2025-02-13 12:57:06 +05:30
Harshal Sheth
7472c535d3
feat(ci): add pytest hooks for updating golden files (#12581) 2025-02-12 15:32:31 -08:00
Mayuri Nehate
2323affde1
feat(ingest): add display name for dynamodb tables (#12534) 2025-02-10 17:54:56 +05:30
Mayuri Nehate
180087f992
fix(ingest): remove duplicate mcps,more typing (#12557) 2025-02-07 14:21:31 +05:30
Jaesung Ryu
06bee0d7c0
feat(ingest/mongodb) re-order aggregation logic (#12428)
Co-authored-by: Kade Ryu <kade@wrtn.io>
2025-02-05 11:25:09 -08:00
Sergio Gómez Villamor
32b654ce20
feat(powerbi): Report to Dashboard lineage (#12451) 2025-02-05 11:27:48 +01:00
Brock Griffey
ac13f255e6
feat(ingestion/snowflake):adds streams as a new dataset with lineage and properties. (#12318)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2025-02-05 10:48:37 +05:30
Sergio Gómez Villamor
6cb51916ef
tests(kafka-connect): fixes integration tests setup (#12531) 2025-02-04 12:41:13 +01:00
Sergio Gómez Villamor
a7598ca20e
fix(mssql): adds missing containers and browsepathsv2 for dataflow and datajob (#12483) 2025-01-31 09:52:26 +01:00
Tamas Nemeth
4dc9bfcde7
fix(ingest/snowflake): Create all structured propery templates before assignation (#12469) 2025-01-30 13:49:20 -08:00
Andrew Sikowitz
aaaa655c21
feat(dataProcessInstance): Support data process instance entity page and lineage (#12499) 2025-01-30 00:39:56 -08:00
sagar-salvi-apptware
563656c4d5
feat(ingestion/lookml): resolve access notation for LookML Constant (#12277)
Co-authored-by: Siddique Bagwan <siddique.bagwan@acryl.io>
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2025-01-28 12:25:45 +05:30
sagar-salvi-apptware
3e9e6e4fe0
fix(lookml/ingestion): Skip unreferenced or improperly loaded Lookml view files (#12351) 2025-01-27 15:37:31 +05:30
Mayuri Nehate
0f538d8df2
fix(ingest): fix reporting for missing secure view lineage (#12430)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2025-01-24 14:51:52 +05:30
Sergio Gómez Villamor
3471857893
refactor(powerbi): dashboards, reports and independent_datasets accessed by id (#12431) 2025-01-23 08:41:46 +01:00
Harshal Sheth
ac44f4a063
feat(ingest/mssql): populate dataTransformLogic aspect for stored procs (#12244) 2025-01-22 12:29:09 -08:00
Tamas Nemeth
7eab2eb8d9
fix(ingest/fivetran): Add way to not add schema to the destination/source urn. (#12314) 2025-01-20 13:38:12 +01:00
Aseem Bansal
262dd76518
dev: remove black in favor of ruff for formatting (#12378) 2025-01-18 15:06:20 +05:30
Aseem Bansal
76e46b89db
dev(ingest): move modules from isort,flake8 to ruff (#12373) 2025-01-17 21:38:29 +05:30
Sergio Gómez Villamor
b7b541ca7d
feat(tableau): fine-grained page size (#12354) 2025-01-16 16:43:46 +01:00
Sergio Gómez Villamor
d99b97a413
chore(tableau): metrics to understand pagination in get_connection_objects queries (#12329) 2025-01-16 10:14:33 +01:00
Andrew Sikowitz
ebbadf65ad
feat(ingest/snowflake): Support ingesting snowflake tags as structured properties (#12285) 2025-01-14 01:50:14 -08:00
Mayuri Nehate
b86bbf7262
fix(ingest/tableau): retry on auth error for special case (#12264) 2025-01-06 12:14:59 +05:30
sagar-salvi-apptware
7f64ffd2f7
test(ingest/athena): add connector integration tests (#12256) 2025-01-02 18:44:45 +05:30
sagar-salvi-apptware
a814cb389f
fix(ingest/bigquery): All View generation when queries_v2 is turned off (#12181)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2025-01-02 17:59:22 +05:30
sagar-salvi-apptware
b79857fd94
fix(ingest/sql-common): sql_common to use SqlParsingAggregator (#12220) 2024-12-29 18:52:05 +05:30
Harshal Sheth
d0423547ba
feat(ingest): add parse_ts_millis helper (#12231) 2024-12-27 10:50:28 -08:00
Harshal Sheth
ed8639e401
chore(python): test with python 3.11 (#11280)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2024-12-27 10:46:49 -08:00
Mayuri Nehate
3ca8d09100
feat(ingest/snowflake): support email_as_user_identifier for queries v2 (#12219) 2024-12-27 11:40:00 +05:30
Harshal Sheth
e1998dd371
chore(ingest): remove enable_logging helper (#12222) 2024-12-26 12:34:00 -08:00
Andrew Sikowitz
f4b33b59d1
fix(ingest/mode): Handle 204 response and invalid json (#12156)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2024-12-24 11:33:06 -08:00
Sergio Gómez Villamor
d88e6c9977
fix(tableau): fixes wrong argument when reauthenticating (#12216) 2024-12-24 16:03:36 +01:00
Mayuri Nehate
4d990b06bd
fix(ingest/snowflake): always ingest view and external table ddl lineage (#12191) 2024-12-24 18:14:51 +05:30
Andrew Sikowitz
09a9b6eef9
feat(ingest/looker): Do not emit usage for non-ingested dashboards and charts (#11647) 2024-12-23 22:39:57 -08:00
Harshal Sheth
21ddb5538d
fix(ingest/mode): move sql logic to view properties aspect (#12196) 2024-12-23 13:32:49 -08:00
Aseem Bansal
ff262bc65e
Revert "fix(mssql): adds missing containers for dataflow and datajob entities, required for browse paths v2 generation" (#12201) 2024-12-22 12:54:18 +00:00