yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-07-19 07:30:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
12,004 Commits 200 Branches 121 Tags
Commit Graph

65 Commits

Author SHA1 Message Date
Harshal Sheth
6a8fca59f1
feat(ingest): use plugin system based on Python extras (#2224) 2021-03-11 13:41:05 -08:00
Harshal Sheth
656d9c771b
feat(ingest): support Postgres PostGIS extensions (#2136) 2021-02-25 09:05:16 -08:00
Harshal Sheth
ac064584ae
refactor(ingest): cleanup configuration models (#2134) 2021-02-23 15:55:31 -08:00
Harshal Sheth
76e0594b8b
feat(ingest): add support for LDAP ingestion (#2122) 2021-02-18 20:05:39 -08:00
Harshal Sheth
38f75be8ad gometa -> datahub 2021-02-15 18:29:27 -08:00
Harshal Sheth
9332e6b878 Add isort to CI 2021-02-15 18:29:27 -08:00
Harshal Sheth
df3e3da45b More autofixes 2021-02-15 18:29:27 -08:00
Harshal Sheth
7ca018aaa4 Remove unused imports 2021-02-15 18:29:27 -08:00
Shirshanka Das
7e0a19c92f bumping up test coverage to 80% 2021-02-15 18:29:27 -08:00
Harshal Sheth
d801c3e1c4 Handle missing kafka schemas elegantly 2021-02-15 18:29:27 -08:00
Harshal Sheth
eaac303610 Reduce mypy errors 2021-02-15 18:29:27 -08:00
Harshal Sheth
9ea03820c4 Enable bare pytest 2021-02-15 18:29:27 -08:00
Harshal Sheth
c7892ada4c Codegen avro + datahub kafka sink (#3) 
* Add codegen

* New architecture + setup file -> console pipeline

* Cleanup source loader

* Basic Kafka metadata source

* Kafka source and extractor

* Add kwargs construct interface

* Fix kafka source unit test

* start working on pipeline test

* kafka datahub sink

* Make myself a profile

* Ingest to datahub from kafka

* Update codegen

* Add restli transport

* Fix bug in restli conversion
 2021-02-15 18:29:27 -08:00
Shirshanka Das
b59a62fa1c setting modest coverage targets 2021-02-15 18:29:27 -08:00
Shirshanka Das
128781942d Firstdrop of ingest (#1) 2021-02-15 18:29:27 -08:00