945 Commits

Author SHA1 Message Date
Teppo Naakka
b223281305
feat(ingest/powerbi): powerbi dataset profiling (#9355) 2024-06-28 14:50:08 -07:00
Harshal Sheth
f4be88d0a9
feat(ingest): set pipeline name in system metadata (#10190)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2024-06-27 15:00:35 -07:00
Harshal Sheth
fa2ab1bcee
fix(ingest): add status aspect to dataProcessInstance (#10757) 2024-06-27 12:07:28 -07:00
Harshal Sheth
0d677e4992
fix(ingest/snowflake): fix column batcher (#10781) 2024-06-25 22:21:54 -07:00
Harshal Sheth
0dc0bc5761
feat(ingest/snowflake): performance improvements (#10746) 2024-06-25 14:46:55 -07:00
Harshal Sheth
2d727a960b
feat(ingest/snowflake): support more than 10k views in a db (#10718) 2024-06-18 07:37:39 +02:00
Rajasekhar-Vuppala
a2d8a099d8
feat(ingest/vertica): use 3 part naming (#10636) 2024-06-14 17:04:55 -07:00
Harshal Sheth
62c6704f69
feat(ingest/snowflake): refactor + parallel schema extraction (#10653) 2024-06-14 13:23:07 -07:00
Shubham Jagtap
3479d070fe
fix(ingestion/sigma): Fix multiple requests http errors (#10616) 2024-06-14 11:30:45 -07:00
sagar-salvi-apptware
cc8389e431
fix(ingest/kafka-connect): Add lineage extraction for BigQuery Sink Connector in Kafka Connect source (#10647) 2024-06-14 18:51:04 +05:30
sagar-salvi-apptware
d69966074a
fix(ingest/bigquery): Map BigQuery policy tags to datahub column-level tags (#10669) 2024-06-14 16:43:12 +05:30
Harshal Sheth
25d48d2d09
fix(ingest/fivetran): fix fivetran bigquery support (#10693) 2024-06-13 11:26:47 -07:00
Harshal Sheth
3a72d92493
feat(ingest/dbt): include package_name in dbt custom props (#10652) 2024-06-12 15:07:42 +02:00
Shubham Jagtap
05aee03f3f
perf(ingestion/fivetran): Connector performance optimization (#10556) 2024-06-11 20:19:57 -07:00
aabharti-visa
8a905774f7
feat(ingestion/kafka)-Add support for ingesting schemas from schema registry (#10612) 2024-06-11 14:00:12 +02:00
Harshal Sheth
e842161849
feat(ingest): add fast query fingerprinting (#10619) 2024-06-05 13:47:44 -07:00
Eric L (CCCS)
c04b3bc2e4
fix(ingest/iceberg): update iceberg source to support newer versions of pyiceberg at runtime (#10614) 2024-06-04 09:45:29 -07:00
Harshal Sheth
db965d61ea
fix(ingest/dbt): only generate one subtype (#10615) 2024-05-29 17:11:34 -07:00
Harshal Sheth
37bc423b50
feat(ingest): enable stateful ingestion safety threshold (#10516) 2024-05-29 12:01:04 -07:00
Paul Rogalinski-Pinter
1c1450e1d8
fix(ingest/metabase): Fix for query template expressions and invalid URNs for Text Cards (#10381)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-05-24 11:35:29 -07:00
sid-acryl
e361d28309
feat(ingestion/looker): ingest explore tags into the DataHub (#10547) 2024-05-23 18:36:43 +05:30
dushayntAW
92780e607c
fix(ingest/unity-catalog) upstream lineage for hive_metastore external table with s3 location (#10546) 2024-05-23 18:34:56 +05:30
Harshal Sheth
7a519ac73c
fix(ingest/dbt): resolve more dbt ephemeral node lineage gaps (#10553) 2024-05-22 12:44:54 -07:00
sid-acryl
666de9e4e6
fix(ingestion/powerbi): Databricks support for table lineage (#10416) 2024-05-22 18:57:45 +05:30
Tamas Nemeth
f831518328
fix(ingest/mode): Adding Dashboards into containers (#10563) 2024-05-22 07:38:57 +02:00
sid-acryl
40d2ae3b78
fix(ingestion/powerbi): handle special character #(tab) in native query parsing (#10520) 2024-05-20 15:20:39 +05:30
Sergio Gómez Villamor
0059960720
feat(ingestion/glue): delta schemas (#10299)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2024-05-17 14:21:35 +02:00
Harshal Sheth
3d5735cbc5
chore(ingest): run pyupgrade for python 3.8 (#10513) 2024-05-15 22:31:05 -07:00
sid-acryl
c55c12c918
fix(ingestion/looker): deduplicate the view field (#10482)
Co-authored-by: Pedro Silva <pedro@acryl.io>
2024-05-15 11:25:07 -07:00
richenc
8e5f17b131
feat(ingest/tableau): support platform instance mapping based off database server hostname (#10254)
Co-authored-by: Richie Chen <richie.chen@hulu.com>
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
2024-05-15 08:23:49 -07:00
dushayntAW
a164b70e1d
chore(ingest/presto-on-hive) Set enable_properties_merge to True by default (#10469) 2024-05-15 18:57:13 +05:30
dushayntAW
bfd3c4bf1c
fix(ingestion/kafka-connect): failure on multiple env substitutes (#10443) 2024-05-09 17:59:55 +05:30
Egemen Berk Galatalı
b1b7cedd8d
feat(ingest/tableau): Fetch Upstreams From Columns (#9874) 2024-05-08 14:21:07 -07:00
dushayntAW
96061be564
fix(ingestion/salesforce): handle the label with none value scenario (#10446) 2024-05-08 14:11:50 +05:30
Tamas Nemeth
897e648eae
fix(ingest/mode): Improve query lineage (#10284)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-05-07 22:02:37 -07:00
Mayuri Nehate
f6627efe71
fix(ingest/snowflake): add more reporting for usage aggregation, handle lineage errors (#10279)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-05-07 08:42:39 -07:00
Shubham Jagtap
ddb38e7448
fix(ingestion/tableau): Fix tableau custom sql lineage gap (#10359) 2024-05-07 13:47:56 +05:30
Harshal Sheth
1dae37a8ed
fix(ingest/bigquery): remove last modified timestamp fallback (#10431) 2024-05-06 16:30:04 -07:00
Ellie O'Neil
d82750b891
DynamoDB IAM auth (#10419) 2024-05-02 16:02:40 -07:00
Pablo Osinaga
0b99145797
feat(metabase): add stateful ingestion (#10360) 2024-05-02 12:04:18 -07:00
sid-acryl
0e795a6614
fix(ingestion/looker): fix lineage for dimension group column (#10382) 2024-05-02 11:48:11 -07:00
Shubham Jagtap
77045f9a7f
perf(ingestion/fivetran): Connector performance optimization (#10346) 2024-04-30 09:44:14 -07:00
Shubham Jagtap
ca2a10e36f
fix(ingestion/qlik): Unable to ingest more than ten spaces (#10228)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-04-23 16:04:20 -07:00
dushayntAW
b9a34fe90b
fix(ingest/salesforce): escape markdown char for multiline description (#10351) 2024-04-23 17:05:55 +05:30
Shubham Jagtap
934ab03d16
test(ingestion/sigma): Add integration test cases (#10356) 2024-04-22 16:19:01 -07:00
Pablo Osinaga
62c7ac706b
feat(ingest/metabase): add ability to exclude other users collections (#10330) 2024-04-19 12:53:17 -07:00
Harshal Sheth
7d31420b69
feat(ingest): materialize terms produced by ingestion (#10249) 2024-04-18 10:48:16 -07:00
Mayuri Nehate
529710ab9d
fix(ingest/databricks): handle and report config parse failure, updat… (#10261) 2024-04-17 12:14:16 -07:00
Tamas Nemeth
d463a16b49
chore(ingest/presto-on-hive): Renaming presto-on-hive to hive-metastore source (#10278) 2024-04-16 23:35:16 +02:00
dushayntAW
20e2cc7eca
fix(ingest/csv): add support multiple ownership type for the same dataset (#10287) 2024-04-15 20:15:08 +05:30