Claudio Benfatto
|
f944a9ba05
|
fix(ingest): enforce correct behaviour for commit policy (#4092)
|
2022-02-08 23:21:23 -08:00 |
|
Gabe Lyons
|
aa253f5b3b
|
feat(deletes): add run commands (list, show, rollback) to datahub ingest (#2960)
|
2021-07-29 20:04:40 -07:00 |
|
Harshal Sheth
|
79f60d8b8a
|
refactor(ingest): remove deprecated methods and warn on deprecated import (#2797)
|
2021-06-29 11:43:43 -07:00 |
|
Harshal Sheth
|
6d875b8241
|
test(ingest): ensure transformer registry works for aliases (#2572)
|
2021-05-17 15:08:49 -07:00 |
|
Harshal Sheth
|
a47400f18e
|
build(ingest): use gradle in commands + docs (#2531)
|
2021-05-11 19:03:20 -07:00 |
|
Harshal Sheth
|
71933a9f31
|
test(ingest): rename TestSource -> FakeSource (#2481)
|
2021-04-30 20:54:07 -07:00 |
|
Harshal Sheth
|
d415234a8c
|
fix(ingest): fields with defaults should be optional (#2461)
|
2021-04-26 16:45:48 -07:00 |
|
Thomas Larsson
|
7869a8f142
|
feature(ingestion): Adding the concept of transformers (#2411)
Fixes: #2410
Co-authored-by: thomas.larsson <thomas.larsson@klarna.com>
|
2021-04-18 11:15:05 -07:00 |
|
Harshal Sheth
|
bd78b84bd3
|
feat(ingest): start airflow integration + metadata builders (#2331)
|
2021-04-05 19:11:28 -07:00 |
|
Harshal Sheth
|
ac064584ae
|
refactor(ingest): cleanup configuration models (#2134)
|
2021-02-23 15:55:31 -08:00 |
|
Harshal Sheth
|
dfe00bfee8
|
feat(ingest): improve error reporting for pipelines (#2121)
|
2021-02-18 11:15:13 -08:00 |
|
Harshal Sheth
|
38f75be8ad
|
gometa -> datahub
|
2021-02-15 18:29:27 -08:00 |
|
Harshal Sheth
|
f807bb44b8
|
Use type + config everywhere
|
2021-02-15 18:29:27 -08:00 |
|
Harshal Sheth
|
df3e3da45b
|
More autofixes
|
2021-02-15 18:29:27 -08:00 |
|
Harshal Sheth
|
7ca018aaa4
|
Remove unused imports
|
2021-02-15 18:29:27 -08:00 |
|
Harshal Sheth
|
43d5fac494
|
Black
|
2021-02-15 18:29:27 -08:00 |
|
Harshal Sheth
|
107cf043d9
|
Add a create method to pipeline
|
2021-02-15 18:29:27 -08:00 |
|
Harshal Sheth
|
a87161cad7
|
Run black formatting on tests
|
2021-02-15 18:29:27 -08:00 |
|
Harshal Sheth
|
8ca8ef2d23
|
Fix kafka tests
|
2021-02-15 18:29:27 -08:00 |
|
Harshal Sheth
|
c7892ada4c
|
Codegen avro + datahub kafka sink (#3)
* Add codegen
* New architecture + setup file -> console pipeline
* Cleanup source loader
* Basic Kafka metadata source
* Kafka source and extractor
* Add kwargs construct interface
* Fix kafka source unit test
* start working on pipeline test
* kafka datahub sink
* Make myself a profile
* Ingest to datahub from kafka
* Update codegen
* Add restli transport
* Fix bug in restli conversion
|
2021-02-15 18:29:27 -08:00 |
|
Shirshanka Das
|
9e61220132
|
checking in testing fixtures. docker still not working
|
2021-02-15 18:29:27 -08:00 |
|