yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-07-12 19:49:36 +00:00
Code Issues Packages Projects Releases Wiki Activity
6,139 Commits 194 Branches 118 Tags
Commit Graph

215 Commits

Author SHA1 Message Date
Harshal Sheth
38f75be8ad gometa -> datahub 2021-02-15 18:29:27 -08:00
Harshal Sheth
0063c04460 gometa-ingest -> datahub ingest 2021-02-15 18:29:27 -08:00
Harshal Sheth
b91d0cf63b Add bigquery and refactor others 2021-02-15 18:29:27 -08:00
Harshal Sheth
d12497a3ff Add postgres source 2021-02-15 18:29:27 -08:00
Harshal Sheth
3678368ed3 Prune deps 2021-02-15 18:29:27 -08:00
Harshal Sheth
9332e6b878 Add isort to CI 2021-02-15 18:29:27 -08:00
Harshal Sheth
0660991fb8 More python 3.6 compat 2021-02-15 18:29:27 -08:00
Harshal Sheth
75b01806ef Use find_packages() in setup 2021-02-15 18:29:27 -08:00
Shirshanka Das
9c6ea3b9e9 Adding 3.9 to setup.py classifiers 2021-02-15 18:29:27 -08:00
Shirshanka Das
024745bcc4 removing py 3.6, adding 3.9 for ci 2021-02-15 18:29:27 -08:00
Harshal Sheth
c7892ada4c Codegen avro + datahub kafka sink (#3) 
* Add codegen

* New architecture + setup file -> console pipeline

* Cleanup source loader

* Basic Kafka metadata source

* Kafka source and extractor

* Add kwargs construct interface

* Fix kafka source unit test

* start working on pipeline test

* kafka datahub sink

* Make myself a profile

* Ingest to datahub from kafka

* Update codegen

* Add restli transport

* Fix bug in restli conversion
 2021-02-15 18:29:27 -08:00
Harshal Sheth
4fb673925c Start using avro producer 2021-02-15 18:29:27 -08:00
Shirshanka Das
1ddbdee60c Support for SQL databases (MySQL + MS-SQL) (#2) 
* adding sql source + mysql

* adding sql support

* MSSQL support, basic integration test

* file sink and pipeline context
 2021-02-15 18:29:27 -08:00
Shirshanka Das
faf472aa64 adding some TODOs 2021-02-15 18:29:27 -08:00
Shirshanka Das
128781942d Firstdrop of ingest (#1) 2021-02-15 18:29:27 -08:00