182 Commits

Author SHA1 Message Date
RyanHolstien
530b9a6540
fix(jackson): increase max serialized string length default (#8053) 2023-05-23 11:38:58 -05:00
Hyejin Yoon
b04d59e13d
docs: add tips on language switchable tap on docs (#7984) 2023-05-08 18:05:22 -07:00
david-leifker
cd05f5b174
feat(schema-registry): replace confluent schema registry (#7930)
Co-authored-by: Pedro Silva <pedro@acryl.io>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: Ryan Holstien <ryan@acryl.io>
2023-05-01 13:18:41 -05:00
david-leifker
2ffa39f009
Logging and test models fixes (#7884) 2023-04-21 15:45:20 -05:00
RyanHolstien
0d5873db2a
feat(patch): patch support for flow info and job info and refactor patchbuilders for java sdk (#7495)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
Co-authored-by: David Leifker <david.leifker@acryl.io>
2023-04-13 15:46:35 -05:00
RyanHolstien
cbf0809c4c
chore(snakeyaml): upgrade to snakeyaml 2 (#7786)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-04-13 12:01:51 -05:00
david-leifker
697e8e2647
fix(misc): misc fixes (#7633) 2023-03-21 19:42:50 +05:30
david-leifker
4a641fb9d3
chore(ci): update command to run smoke tests (#7284) 2023-02-09 21:26:29 +05:30
mohdsiddique
90a1da3b25
fix(test): spark-lineage # build task as dependency of integrationTest (#7189)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-01-31 19:50:00 -08:00
(((((Al)))))
48080b2d29
fix(ingest): add message proto check (#7130)
Co-authored-by: Tino Lyu <tinol@ziprecruiter.com>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-01-25 20:37:45 -06:00
Fully
a6a597cc81
feat(ingest/spark): add removing partition pattern in spark lineage (#6605)
Co-authored-by: John Joyce <john@acryl.io>
2023-01-24 09:32:11 +01:00
seoju
65c275979b
fix(ingest): fix logic error of google protobuf wrapper type. (#7076)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-01-20 07:33:00 -06:00
seoju
aba3881903
feat(ingest): add import descriptions of two or more nested messages (#6959)
Co-authored-by: 서주현[G플레이스데이터개발] <juhyun.seo@navercorp.com>
2023-01-09 16:22:37 -08:00
danielli-ziprecruiter
a6470fc267
feat(ingestion): spark - support lineage for delta lake writes (#6834) 2022-12-22 12:13:32 +01:00
danielli-ziprecruiter
ca6c3b949b
fix(ingest): exclude ztsd from uber jar to prevent jni conflicts with spark (#6787)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-12-20 21:54:56 -06:00
danielli-ziprecruiter
7ca21a8e7c
fix(ingest): prevent NullPointerException when non-jdbc SaveIntoDataSourceCommand (#6803) 2022-12-19 08:41:06 -08:00
david-leifker
3758bcbe34
fix(docs): update examples for protobuf (#6681) 2022-12-07 16:48:01 -06:00
Shirshanka Das
8e05fdf516
fix(datahub-protobuf): fix missing httpclient dependency (#6672) 2022-12-07 07:28:17 -06:00
david-leifker
2de9d3d5bf
fix(logging): Remove lombok as source of slf4j-api, convert to compileOnly where possible (#6616) 2022-12-04 19:57:47 -08:00
david-leifker
ef9a280ba4
fix(verions): Update opentelemetry and updates from pr-5239 (#6563) 2022-11-29 10:59:38 -06:00
david-leifker
a2dc229c62
fix(tests): Misc updates for tests, auth log level, and quickstart (#6491) 2022-11-29 08:44:55 -08:00
david-leifker
33fd876421
feat(elasticsearch): Updates to elasticsearch configuration, dao, and tests (#6269) 2022-11-15 18:03:11 -08:00
Tamas Nemeth
8cfcefb19c
fix(spark-lineage): smoke test fixes, M1 support (#6372) 2022-11-05 11:37:50 -07:00
Tamas Nemeth
ff3a9dd4b9
fix(spark-lineage): Rename table.platform to table.hive_platform_alias (#6311) 2022-10-28 16:59:17 -07:00
danielli-ziprecruiter
7b2a76dd4c
feat(spark-lineage): make platform for spark lineage inferred and configurable (#6251)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-10-27 21:59:18 +02:00
Mayuri Nehate
38a2b211af
docs(spark): add configuration instructions for databricks (#6206) 2022-10-14 16:55:07 -07:00
david-leifker
3dc25c14e0
test(KafkaEmitter): Enable ability to run test locally (#6123) 2022-10-06 13:34:11 -07:00
david-leifker
928f29438a
build: Update JNA for M1 Mac (#6116) 2022-10-05 14:53:38 -07:00
leifker
203a6ff57b
refactor(java11) - convert most modules to java 11 (#5836)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-25 08:39:22 -07:00
RyanHolstien
0728656fbf
feat(patch): initial support of json patch style semantics in MCPs (#5901)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-22 23:52:18 -07:00
Shirshanka Das
8e8fb51ce3
feat(cli): extend put command to support platform creation (#5990) 2022-09-20 11:52:22 -07:00
Shirshanka Das
41746690fd
feat(protobuf): support for custom platform, subtypes, misc improvements (#5973) 2022-09-18 18:04:47 -07:00
Shirshanka Das
083ab9bc0e
feat(jars): better jar versioning for datahub-client, spark-lineage and protobuf (#5883) 2022-09-08 23:36:15 -07:00
Shirshanka Das
fe30f54689
feat(datahub-protobuf): add support for shadow jar, publish (#5882) 2022-09-08 22:10:36 -07:00
Tamas Nemeth
caec2ed235
fix(ingest): spark-lineage - Adding additional debug logs to spark lineage (#5772) 2022-09-04 22:43:03 -07:00
Harshal Sheth
762bcbb168
chore(docker): make curl/wget commands quiet in docker (#5819) 2022-09-04 22:39:08 -07:00
Mugdha Hardikar
fe0773278b
Create application setup on sqlevent start (#5657) 2022-08-17 16:02:37 +02:00
Mugdha Hardikar
c7f477813c
feat(datahub-client): add java file emitter (#5578)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-08-07 07:21:44 -07:00
Mugdha Hardikar
d8b2350724
feat(spark-lineage, java-emitter): Support ssl cert disable verification functionality (#5488)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-07-26 18:06:03 -07:00
Tamas Nemeth
54ff4524dd
docs: spark-lineage - configuration details for Amazon EMR (#5459) 2022-07-21 08:57:26 -07:00
leifker
e9c943f3b9
fix(protobuf) Set undeprecated ownership type & fix case sentitive urn corpGroup (#5425) 2022-07-18 15:56:43 -07:00
Harshal Sheth
fdea718e38
build: improve comprehensiveness of gradle clean (#5003) 2022-07-15 15:38:32 +05:30
Mugdha Hardikar
369c65522c
fix(datahub-client): Fix kafka config issue (#5403) 2022-07-15 15:38:00 +05:30
Mugdha Hardikar
8358ea8e3e
feat(datahub-client): add java kafka emitter (#5074) 2022-06-20 17:32:57 -07:00
RyanHolstien
3be8afd72f
chore(guava): force version of guava in client jars per #5134 (#5153) 2022-06-14 13:46:00 -07:00
MugdhaHardikar-GSLab
ccf82229d5
feat(spark-lineage): coalesce spark jobs (#5077) 2022-06-03 16:32:22 +02:00
Shirshanka Das
eca310fb05
feat(ci): datahub-client - add workflow, fix build (#5052) 2022-05-31 22:40:31 -07:00
MugdhaHardikar-GSLab
1479c35bfd
fix(spark-lineage): remove need for sparksession.stop call (#4940)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-05-29 09:07:26 -07:00
MugdhaHardikar-GSLab
96ef7e55ae
feat(spark-lineage): support for persist API (#4980) 2022-05-23 20:53:57 +02:00
MugdhaHardikar-GSLab
fc8a8030e4
fix(datahub-client): support utf8 encoding (#4878)
fixes #4700
2022-05-20 10:45:53 -07:00