945 Commits

Author SHA1 Message Date
ryota-cloud
9f8753bcb2
feat(ingestion) Adding vertexAI ingestion source (v2 - experiment and experiment run) (#12836) 2025-03-21 14:06:17 -07:00
Peter Wang
6914df3721
feat(ingestion/superset): add timeout values to config to prevent hanging queries from blocking ingestion (#12884) 2025-03-21 08:08:56 -07:00
Mayuri Nehate
8323bc3910
fix(ingest/dremio): simplify and fix build source map (#12908) 2025-03-21 11:31:16 +05:30
skrydal
1185ba8121
feat(ingestion/iceberg): Refactor iceberg source to use MCPWs instead of MCEs (#12912) 2025-03-20 10:46:08 +01:00
Austin SeungJun Park
41895fe24f
feat(ingest/s3): add table filtering (#12661) 2025-03-20 07:57:43 +01:00
Sergio Gómez Villamor
2aba2e3ed8
fix(powerbi): fixes direction of the dashboard-report lineage (#12881) 2025-03-19 12:25:42 +01:00
Gabe Lyons
ecd9ffd137
feat(openapi): Adding subtype for openapi source (#12873) 2025-03-18 12:35:46 -07:00
Peter Wang
cffc6d4693
feat(ingestion/superset): superset column level lineage (#12786) 2025-03-14 08:02:54 -07:00
Deepak Garg
eb17f80e8a
feat(ingest/hive): identify partition columns in hive tables (#12833) 2025-03-13 11:37:52 -07:00
ryota-cloud
0e62e8c77a
feat(ingestion) Adding vertexAI ingestion source (v1 - model group and model) (#12632) 2025-03-13 11:02:15 -07:00
Mayuri Nehate
f9d71d67a0
feat(ingest/salesforce): include formula in in field description (#12840) 2025-03-13 22:00:06 +05:30
Hyejin Yoon
86f4b805cf
feat(ingest/mlflow): update dpi to use edge for lineage (#12861) 2025-03-13 10:05:00 +09:00
Hyejin Yoon
8bf1f718da
feat(ingestion/mlflow): improve mlflow connector to pull run and experiments (#12587) 2025-03-11 07:55:48 +09:00
Peter Wang
48b6581f12
fix(ingest/superset): fixed iterate over int error for building urns (#12807)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-03-07 15:41:27 -08:00
Harshal Sheth
55fbb71a53
fix(ingest/oracle): refresh golden files (#12818) 2025-03-07 12:38:39 -08:00
Jonny Dixon
fcabe88962
fix(ingestion/oracle): Improved foreign key handling (#11867)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2025-03-06 14:32:03 +00:00
Jonny Dixon
a700448bad
feat(ingestion/business-glossary): Automatically generate predictable glossary term and node URNs when incompatible URL characters are specified in term and node names. (#12673) 2025-03-06 14:30:10 +00:00
Harshal Sheth
de60ca30e8
fix(ingest): enable fuzzy case resolution for oracle sql (#12778) 2025-03-05 10:28:29 -08:00
Sergio Gómez Villamor
69981675a5
feat(mssql): adds subtypes aspect for dataflow and datajobs (#12775) 2025-03-05 17:11:04 +01:00
Sergio Gómez Villamor
85d3a9d31d
feat(okta): custom properties for okta user (#12773) 2025-03-05 15:53:01 +01:00
Peter Wang
9e7f48278a
feat(ingestion/superset): ownership info for charts, dashboards and datasets (#12750) 2025-03-04 12:28:24 -08:00
skrydal
3e1b20c727
feat(ingestion/iceberg): Several improvements to iceberg connector (#12744) 2025-03-01 01:30:07 +01:00
Peter Wang
fc26c1cfb5
feat(ingestion/superset): superset dataset lineage for metadata ingestion (#12679)
Co-authored-by: Shuixi Li <llance@users.noreply.github.com>
Co-authored-by: Gabe Lyons <gabe.lyons@acryl.io>
2025-02-28 07:29:25 -08:00
Aseem Bansal
9c3bd34995
dev: enable ruff rule (#12749) 2025-02-28 17:49:52 +05:30
Aseem Bansal
0fc074b50a
dev: enable ruff rule (#12742) 2025-02-28 10:59:06 +05:30
Aseem Bansal
2101a50e83
dev: immutable args remove (#12735) 2025-02-27 15:38:47 +05:30
Rasnar
5f5e395c90
Fix platform instance support on Druid ingestion (#12716)
Co-authored-by: rasnar <11248833+Rasnar@users.noreply.github.com>
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-02-26 08:36:06 +01:00
Sergio Gómez Villamor
c980cbd8f2
tests(druid): integration tests for druid ingestion (#12717)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2025-02-25 08:08:45 +01:00
Harshal Sheth
8dfd8fbbaf
feat(ingest): migrate Cassandra source to new SDK (#12695) 2025-02-24 11:57:51 -08:00
Mayuri Nehate
fbed37b374
fix(ingest/snowflake): update logic to identify hybrid tables (#12672) 2025-02-20 11:23:57 +05:30
mihai103
dcf1576c30
fix(tableau): skip ingesting filtered project assets (#12591)
Co-authored-by: Mihai Ciocirdel <mihai.ciocirdel@swisscom.com>
Co-authored-by: Jay <159848059+jayacryl@users.noreply.github.com>
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-02-14 09:25:37 +01:00
Sergio Gómez Villamor
b20211536c
feat(okta): adds ingest_groups_users config parameter (#12371)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2025-02-13 12:00:07 +01:00
Mayuri Nehate
e2a98d217e
fix(ingest): pick topics from config for sink connector (#12535) 2025-02-13 12:57:06 +05:30
Harshal Sheth
7472c535d3
feat(ci): add pytest hooks for updating golden files (#12581) 2025-02-12 15:32:31 -08:00
Mayuri Nehate
2323affde1
feat(ingest): add display name for dynamodb tables (#12534) 2025-02-10 17:54:56 +05:30
Mayuri Nehate
180087f992
fix(ingest): remove duplicate mcps,more typing (#12557) 2025-02-07 14:21:31 +05:30
Jaesung Ryu
06bee0d7c0
feat(ingest/mongodb) re-order aggregation logic (#12428)
Co-authored-by: Kade Ryu <kade@wrtn.io>
2025-02-05 11:25:09 -08:00
Sergio Gómez Villamor
32b654ce20
feat(powerbi): Report to Dashboard lineage (#12451) 2025-02-05 11:27:48 +01:00
Brock Griffey
ac13f255e6
feat(ingestion/snowflake):adds streams as a new dataset with lineage and properties. (#12318)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2025-02-05 10:48:37 +05:30
Sergio Gómez Villamor
6cb51916ef
tests(kafka-connect): fixes integration tests setup (#12531) 2025-02-04 12:41:13 +01:00
Sergio Gómez Villamor
a7598ca20e
fix(mssql): adds missing containers and browsepathsv2 for dataflow and datajob (#12483) 2025-01-31 09:52:26 +01:00
Tamas Nemeth
4dc9bfcde7
fix(ingest/snowflake): Create all structured propery templates before assignation (#12469) 2025-01-30 13:49:20 -08:00
Andrew Sikowitz
aaaa655c21
feat(dataProcessInstance): Support data process instance entity page and lineage (#12499) 2025-01-30 00:39:56 -08:00
sagar-salvi-apptware
563656c4d5
feat(ingestion/lookml): resolve access notation for LookML Constant (#12277)
Co-authored-by: Siddique Bagwan <siddique.bagwan@acryl.io>
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2025-01-28 12:25:45 +05:30
sagar-salvi-apptware
3e9e6e4fe0
fix(lookml/ingestion): Skip unreferenced or improperly loaded Lookml view files (#12351) 2025-01-27 15:37:31 +05:30
Mayuri Nehate
0f538d8df2
fix(ingest): fix reporting for missing secure view lineage (#12430)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2025-01-24 14:51:52 +05:30
Sergio Gómez Villamor
3471857893
refactor(powerbi): dashboards, reports and independent_datasets accessed by id (#12431) 2025-01-23 08:41:46 +01:00
Harshal Sheth
ac44f4a063
feat(ingest/mssql): populate dataTransformLogic aspect for stored procs (#12244) 2025-01-22 12:29:09 -08:00
Tamas Nemeth
7eab2eb8d9
fix(ingest/fivetran): Add way to not add schema to the destination/source urn. (#12314) 2025-01-20 13:38:12 +01:00
Aseem Bansal
262dd76518
dev: remove black in favor of ruff for formatting (#12378) 2025-01-18 15:06:20 +05:30