14 Commits

Author SHA1 Message Date
Shirshanka Das
c5fc2ea798
fix(ingest): looker - deps, column level lineage fixes (#6271) 2022-10-24 08:31:48 +02:00
Harshal Sheth
55cb34e5d4
test(ingest): limit GMS retries in test (#5509) 2022-07-29 15:11:31 +05:30
Claudio Benfatto
aeefde4fa1
feat(ingestion): Kafka stateful ingestion (#4028)
* test: test stateful ingestion for kafka

test: some more advancement

test: some improvements

refactoring

* refactor: remove some linter modifications

* tests: add unit tests for kafka state

* refactor: minor changes

* tests: improve test coverage

* fix: fix naming

* style: fix format with black

* fix: fix broken test

* revert: revert smoke tests to master

* feat: add reporting to kafka source

* tests: add smoke tests for kafka reporting

* revert: revert changes to the smoke tests

* test: add kafka integration test for stateful ingestion

* docs: update documentation on kafka source

* fix: return empty string when no platform instance

* revert: remove unwanted file

* fix: solve problem with platform instance

* chore: use console sink instead of file

* fix: disable complexity check for _extract_record

* fix: remove if condition in get_platform_instance_id

* chore: remove unneeded integration test

* test: test platform instance in kafka source unit tests
2022-02-15 07:18:36 -08:00
Kevin Hu
d3081f4807
feat(ingestion): anonymous usage stats (#3668) 2021-12-07 08:57:12 -08:00
Harshal Sheth
b0b275ee34
test(profiler): make profiling tests more comprehensive (#3513) 2021-11-07 18:53:53 -08:00
Harshal Sheth
d66381451a
feat(ingest): refactor mce comparison and add pytest update golden files option (#2812) 2021-06-30 16:53:20 -07:00
Harshal Sheth
a11329d5b8
refactor(ingest): update test harness to use a compose file per test (#2392) 2021-04-13 17:30:24 -07:00
Harshal Sheth
b8462028c3
feat(ingest): various minor fixes (#2246) 2021-03-17 23:05:05 -07:00
Harshal Sheth
4e15feebfd
fix(ingestion): nullable types and timestamp precision (#2212) 2021-03-10 23:29:24 -08:00
Harshal Sheth
9332e6b878 Add isort to CI 2021-02-15 18:29:27 -08:00
Harshal Sheth
a87161cad7 Run black formatting on tests 2021-02-15 18:29:27 -08:00
Harshal Sheth
b759205b52 Start checking files for mysql test 2021-02-15 18:29:27 -08:00
Harshal Sheth
0929c7cb77 Add test to check that classes are not abstract 2021-02-15 18:29:27 -08:00
Harshal Sheth
9ea03820c4 Enable bare pytest 2021-02-15 18:29:27 -08:00