23 Commits

Author SHA1 Message Date
Harshal Sheth
79f844b208
chore(py): add unused-noqa lint rule (#12615) 2025-02-13 08:28:48 -08:00
Harshal Sheth
7472c535d3
feat(ci): add pytest hooks for updating golden files (#12581) 2025-02-12 15:32:31 -08:00
Harshal Sheth
37bb0c9eae
fix(ingest/sql): disable patch checker (#11910) 2024-11-20 13:48:56 -08:00
Harshal Sheth
d34717fd82
fix(ingest): remove default value from DatahubClientConfig.server (#11570) 2024-10-16 13:50:33 -07:00
Harshal Sheth
9dd0c37830
feat(ingest): fix fspath lint error (#9976) 2024-03-04 17:09:22 -08:00
Harshal Sheth
b01416c11b
feat(ingest): add timeout to sqlglot parser (#9868) 2024-02-23 11:11:02 -08:00
Harshal Sheth
9deb7be3fc
fix(ingest): refactor test markers + fix disk space issues in CI (#8938) 2023-10-03 20:17:49 -07:00
Harshal Sheth
4fb77e4a25
fix(ingest): tweak ingestion exit codes (#8418) 2023-07-14 15:47:16 -07:00
Mayuri Nehate
70c9b20c6d
feat(ingest/kafka-connect): add stateful ingestion and platform instance support (#7526) 2023-03-14 13:59:54 -07:00
Shirshanka Das
c5fc2ea798
fix(ingest): looker - deps, column level lineage fixes (#6271) 2022-10-24 08:31:48 +02:00
Harshal Sheth
55cb34e5d4
test(ingest): limit GMS retries in test (#5509) 2022-07-29 15:11:31 +05:30
Claudio Benfatto
aeefde4fa1
feat(ingestion): Kafka stateful ingestion (#4028)
* test: test stateful ingestion for kafka

test: some more advancement

test: some improvements

refactoring

* refactor: remove some linter modifications

* tests: add unit tests for kafka state

* refactor: minor changes

* tests: improve test coverage

* fix: fix naming

* style: fix format with black

* fix: fix broken test

* revert: revert smoke tests to master

* feat: add reporting to kafka source

* tests: add smoke tests for kafka reporting

* revert: revert changes to the smoke tests

* test: add kafka integration test for stateful ingestion

* docs: update documentation on kafka source

* fix: return empty string when no platform instance

* revert: remove unwanted file

* fix: solve problem with platform instance

* chore: use console sink instead of file

* fix: disable complexity check for _extract_record

* fix: remove if condition in get_platform_instance_id

* chore: remove unneeded integration test

* test: test platform instance in kafka source unit tests
2022-02-15 07:18:36 -08:00
Kevin Hu
d3081f4807
feat(ingestion): anonymous usage stats (#3668) 2021-12-07 08:57:12 -08:00
Harshal Sheth
b0b275ee34
test(profiler): make profiling tests more comprehensive (#3513) 2021-11-07 18:53:53 -08:00
Harshal Sheth
d66381451a
feat(ingest): refactor mce comparison and add pytest update golden files option (#2812) 2021-06-30 16:53:20 -07:00
Harshal Sheth
a11329d5b8
refactor(ingest): update test harness to use a compose file per test (#2392) 2021-04-13 17:30:24 -07:00
Harshal Sheth
b8462028c3
feat(ingest): various minor fixes (#2246) 2021-03-17 23:05:05 -07:00
Harshal Sheth
4e15feebfd
fix(ingestion): nullable types and timestamp precision (#2212) 2021-03-10 23:29:24 -08:00
Harshal Sheth
9332e6b878 Add isort to CI 2021-02-15 18:29:27 -08:00
Harshal Sheth
a87161cad7 Run black formatting on tests 2021-02-15 18:29:27 -08:00
Harshal Sheth
b759205b52 Start checking files for mysql test 2021-02-15 18:29:27 -08:00
Harshal Sheth
0929c7cb77 Add test to check that classes are not abstract 2021-02-15 18:29:27 -08:00
Harshal Sheth
9ea03820c4 Enable bare pytest 2021-02-15 18:29:27 -08:00