191 Commits

Author SHA1 Message Date
Jonny Dixon
58d8a802f8 better capture of microbatchexecution and logs 2025-06-03 10:03:48 +01:00
Jonny Dixon
75086ada94 linting 2025-06-03 01:04:02 +01:00
Jonny Dixon
954e369ad7 job lineage 2025-06-02 00:28:55 +01:00
Jonny Dixon
eca5a35613 streaming improvements - adding correlator for inputs and outputs 2025-06-01 22:13:32 +01:00
Jonny Dixon
f59afd7154 streaming improvements - adding correlator for inputs and outputs 2025-06-01 22:12:20 +01:00
Jonny Dixon
2a9b7479c2 spark streaming with passed config 2025-05-31 23:35:58 +01:00
Jonny Dixon
7a4ef72a34
Merge branch 'master' into split-spark-merges-on-tables 2025-05-29 20:50:36 +01:00
Jonny Dixon
cfc05747aa
Openlineage 1 31 0 upgrade - Tamas' changes (#13658)
Co-authored-by: treff7es <treff7es@gmail.com>
2025-05-29 20:50:11 +01:00
david-leifker
4c6672213c
config(gradle): pin jackson via bom (#13617) 2025-05-24 13:42:10 -05:00
Jonny Dixon
2cee79fd4d docs prettier 2025-05-23 20:57:41 +01:00
Jonny Dixon
0446c68e3e add backstop 2025-05-23 20:51:43 +01:00
Jonny Dixon
da44176cfe fixes for spark streaming NPE for generating flow_urns 2025-05-23 20:44:33 +01:00
Jonny Dixon
75f12621c9 standardized job name 2025-05-23 13:08:10 +01:00
Jonny Dixon
c548d9f465 add parameter and docs 2025-05-22 21:31:17 +01:00
Jonny Dixon
f42a9ef920 delta merge extraction 2025-05-22 20:31:47 +01:00
Esteban Gutierrez
52e01bd599
fix(smoke-test): use full quickstart image instead of slim for spark tests (#13543) 2025-05-19 11:04:53 -05:00
Kevin Karch
18d1a3c2e3
fix(docs): broken link in spark docs (#13516) 2025-05-14 16:11:44 -04:00
Hyejin Yoon
f986315582
doc: Acryl to DataHub, datahubproject.io to datahub.com (#13252)
Co-authored-by: Jay <159848059+jayacryl@users.noreply.github.com>
2025-04-28 10:34:33 -04:00
Andrew Sikowitz
d138a64a6a
ci(graphql,workflows): Format .md, .graphql, and workflow .yml files via prettier (#13220) 2025-04-16 16:55:51 -07:00
david-leifker
5ee0b66920
feat(spring): upgrade to SpringBoot 3.4 (#13186) 2025-04-12 15:34:17 -05:00
david-leifker
dd14507d7d
fix(openapi): required fields w/ defaults (#13095) 2025-04-07 10:51:03 -05:00
Chakru
40106be208
build: optimizations for incremental builds and faster CI (#13033)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2025-04-02 11:51:10 +05:30
Chakru
7c1ed744f4
fix(build): build improvements to help with incremental builds (#12823) 2025-03-10 22:43:31 +05:30
Sergio Gómez Villamor
64aaaf126f
fix(model): fixes DashboardContainsDashboard relationship in DashboardInfo aspect (#12433) 2025-02-03 11:33:25 +01:00
david-leifker
462de489d8
refactor(GMS): migrate GMS to Spring boot (#12389) 2025-01-23 10:45:40 -06:00
Tamas Nemeth
3316d407f7
fix(spark): Finegrained lineage is emitted on the DataJob and not on the emitted Datasets. (#11956) 2025-01-06 18:21:24 +01:00
Sergio Gómez Villamor
d2ffeb00a5
tests(datahub-client): new tests for the AvroSchemaConverter (#12087) 2024-12-16 11:26:57 +00:00
Dejan Stojadinović
2291c71c29
build(gradle): version change (Gradle and shadow plugin) (#11999) 2024-12-14 10:04:48 -06:00
Sergio Gómez Villamor
613b433d18
fix(datahub-client): adds missing archiveAppendix to artifactid when publishing (#12112) 2024-12-12 18:11:54 +00:00
Sergio Gómez Villamor
edca8eae99
feat(datahub-client): additionally generates java8 artefacts (#12106) 2024-12-12 14:03:30 +00:00
david-leifker
cb7d687794
fix(datahub-client): prevent unneeded classes in datahub-client jar (#12037) 2024-12-05 11:45:04 -06:00
david-leifker
a004c9293d
fix(ci): fix datahub-client validatePythonEnv (#12023) 2024-12-03 15:47:08 -06:00
Shirshanka Das
7429075ebd
feat(java-sdk): add utils classes to give equivalence with python uti… (#12002) 2024-12-02 23:23:22 -08:00
Chakru
85b42e3ea5
build(coverage): enable code coverage for java and python (#11992)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2024-12-02 19:27:43 -06:00
Jonny Dixon
2f20c529f4
fix(docs): Add spark.datahub.stage_metadata_coalescing to recommended configuration for databricks (#11800) 2024-12-02 10:08:20 -08:00
david-leifker
0ee758c52d
fix(schematron): fix for jdk8 (#11975) 2024-11-27 12:10:23 -06:00
david-leifker
3b00fd7646
fix(shadowJar): fix shadowJar (#11968) 2024-11-26 21:02:28 -06:00
Shirshanka Das
f9cc67dd31
feat(schematron): add java capabilities for schema translation (#11963) 2024-11-26 16:03:22 -06:00
Tamas Nemeth
1f396e87c1
feat(spark): OpenLineage 1.24.2 upgrade (#11830) 2024-11-19 15:13:24 -08:00
skrydal
becda8fd34
docs(ingest): transformer docs and java example (#11859) 2024-11-15 20:19:45 +05:30
ronybony1990
edb87ff2ea
feat(spark-plugin): user should be able to pass custom mcp kafka topic (#11767)
Co-authored-by: Neelab Chaudhuri <nchaudhuri@fanatics.com>
2024-11-05 11:18:15 +01:00
Steffen Grohsschmiedt
4634bbcae0
docs(spark): fix incorrect config option (#11119) 2024-10-31 16:01:25 -07:00
david-leifker
413331933a
fix(spark-lineage-legacy): fix check jar script (#11608) 2024-10-11 19:08:25 -05:00
Aseem Bansal
867007f209
chore: remove obsolete attribute from docker (#11573) 2024-10-09 20:52:50 +05:30
david-leifker
db49067111
build(misc): misc build, config, version updates (#11527) 2024-10-03 21:10:26 -05:00
david-leifker
2ceb8e0934
chore(version): bump protobuf version (#11446) 2024-09-20 14:06:31 -05:00
david-leifker
1d297afe35
fix(ebean): upgrade ebean library (#11379) 2024-09-16 18:37:36 -05:00
david-leifker
d795a5357d
fix(search): fix regression from #10932 (#11309) 2024-09-05 14:09:13 -05:00
eunseokyang
59f9a4ba55
feat(ingest): Support protobuf description for enum field (#11027)
Co-authored-by: 양은석[G플레이스데이터개발] <eunseok.y@navercorp.com>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2024-09-05 10:32:58 -05:00
eunseokyang
e45674852c
fix(ingest/protobuf): Improve String Handling for Multilingual Support in Descriptions (#10975)
Co-authored-by: happyprg <hongsgo@gmail.com>
2024-09-05 10:32:36 -05:00