26 Commits

Author SHA1 Message Date
Sergio Gómez Villamor
8cae980286
tests(ingestion): moving some tests so they are available for sdk users (#13540) 2025-05-19 08:39:53 +02:00
Sergio Gómez Villamor
a8637abfe2
tests(kafka): fixing flaky tests (#13171) 2025-04-22 12:58:47 +02:00
Aseem Bansal
9c3bd34995
dev: enable ruff rule (#12749) 2025-02-28 17:49:52 +05:30
Mayuri Nehate
e2a98d217e
fix(ingest): pick topics from config for sink connector (#12535) 2025-02-13 12:57:06 +05:30
Aseem Bansal
262dd76518
dev: remove black in favor of ruff for formatting (#12378) 2025-01-18 15:06:20 +05:30
sagar-salvi-apptware
cc8389e431
fix(ingest/kafka-connect): Add lineage extraction for BigQuery Sink Connector in Kafka Connect source (#10647) 2024-06-14 18:51:04 +05:30
Harshal Sheth
3d5735cbc5
chore(ingest): run pyupgrade for python 3.8 (#10513) 2024-05-15 22:31:05 -07:00
dushayntAW
bfd3c4bf1c
fix(ingestion/kafka-connect): failure on multiple env substitutes (#10443) 2024-05-09 17:59:55 +05:30
Harshal Sheth
9deb7be3fc
fix(ingest): refactor test markers + fix disk space issues in CI (#8938) 2023-10-03 20:17:49 -07:00
Shubham Jagtap
501522d891
feat(ingest/kafka-connect): Lineage for Kafka Connect > Snowflake (#8811)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-09-22 17:12:48 -07:00
TusharM
e2919cfd06
feat(ingest/kafka-connect): add support for Confluent S3 Sink Connector (#8298) 2023-08-02 15:09:50 +05:30
Andrew Sikowitz
ea3e119210
test(ingest/kafka-connect): Attempt to fix flaky test (#8404) 2023-07-12 00:46:10 -04:00
Andrew Sikowitz
d92cbe99e0
test(ingest/kafka): Configure kafka connect tests for arm64 (#8362) 2023-07-10 12:27:29 -04:00
Harshal Sheth
b0f8c3de1e
refactor(ingest): simplify stateful ingestion provider interface (#8104)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-05-23 12:57:57 -07:00
Harshal Sheth
e99875cac6
chore(ingest): enable flake8 bugbear linting (#7763) 2023-04-10 14:14:42 -07:00
Harshal Sheth
8394dcb538
chore(ingest): change kafka connect mapped ports (#7728) 2023-04-04 18:38:30 +05:30
Mayuri Nehate
70c9b20c6d
feat(ingest/kafka-connect): add stateful ingestion and platform instance support (#7526) 2023-03-14 13:59:54 -07:00
Harshal Sheth
1b889022f0
test(ingest/kafka-connect): make docker setup more reliable (#6902) 2022-12-30 11:31:33 +01:00
Fredrik Sannholm
4dd66be654
feat(ingest/kafka-connect): support MongoSourceConnector (#6416)
Co-authored-by: John Joyce <john@acryl.io>
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-12-05 16:09:58 -05:00
BZ
64c28a2316
feat(ingest/kakfa-connect): support generic connectors (#6173)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-10-25 10:31:31 -07:00
Shirshanka Das
558a65a3c3
fix(ci): fix mysql test and attempt kafka-connect ingestion (#5352) 2022-07-07 08:28:34 -07:00
Aseem Bansal
c942350e3c
feat(ci): split integration tests to reduce run time (#5205) 2022-06-20 21:08:37 +05:30
mayurinehate
467ea7917c
fix(kafka-connect): fix lineage for postgres-like 3-level hierarchy d… (#4375)
* fix(kafka-connect): fix lineage for postgres-like 3-level hierarchy dialects in jdbc source connnector

Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-03-30 20:59:02 -07:00
Harshal Sheth
22cef5f897
refactor(test): replace CliRunner with run_datahub_cmd method (#3746) 2021-12-16 20:07:38 -08:00
mayurinehate
69036acfca
feat(ingest): add bigquery sink connector lineage in kafka connect source (#3590) 2021-11-17 17:18:37 -08:00
mayurinehate
192f0d33a2
feat(ingest): kafka connect source improvements (#3481) 2021-11-03 15:03:05 -07:00