yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-07-30 21:10:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,652 Commits 209 Branches 123 Tags
Commit Graph

3652 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Harshal Sheth
fd9bc09e67 Start adding reporting 2021-02-15 18:29:27 -08:00
Harshal Sheth
0929c7cb77 Add test to check that classes are not abstract 2021-02-15 18:29:27 -08:00
Harshal Sheth
9ea03820c4 Enable bare pytest 2021-02-15 18:29:27 -08:00
Shirshanka Das
063f513997 Update README.md 2021-02-15 18:29:27 -08:00
Shirshanka Das
4e17a24dd4 Adding mysql integration test + source 2021-02-15 18:29:27 -08:00
Shirshanka Das
e03a9e25f8 Update README.md 2021-02-15 18:29:27 -08:00
Harshal Sheth
22a4dbb45f Cleanup kafka sink schema setup 2021-02-15 18:29:27 -08:00
Shirshanka Das
d5f89f8170 adding commented out example to yaml file 2021-02-15 18:29:27 -08:00
Harshal Sheth
d68648670a Add back line that I accidentally removed when testing 2021-02-15 18:29:27 -08:00
Harshal Sheth
4b83fc6591 adding allow deny patterns to sql config 2021-02-15 18:29:27 -08:00
Harshal Sheth
62bb7f012f Quick readme updates 2021-02-15 18:29:27 -08:00
Harshal Sheth
9d741815c5 Also run unit tests in CI 2021-02-15 18:29:27 -08:00
Harshal Sheth
e09528909f python 3.7 compat 2021-02-15 18:29:27 -08:00
Shirshanka Das
9c6ea3b9e9 Adding 3.9 to setup.py classifiers 2021-02-15 18:29:27 -08:00
Shirshanka Das
024745bcc4 removing py 3.6, adding 3.9 for ci 2021-02-15 18:29:27 -08:00
Shirshanka Das
7b12fc9827 support for multiple mce-s in a file 2021-02-15 18:29:27 -08:00
Harshal Sheth
58554725aa Add bootstrap mce 2021-02-15 18:29:27 -08:00
Harshal Sheth
4dc8936f27 Add mypy type checks 2021-02-15 18:29:27 -08:00
Shirshanka Das
5a8bb3cfac adding docker commands 2021-02-15 18:29:27 -08:00
Shirshanka Das
863cc97ea3 simplify docker action 2021-02-15 18:29:27 -08:00
Shirshanka Das
1c05c1b23a simplify docker action 2021-02-15 18:29:27 -08:00
Shirshanka Das
e80ebf180c simplify docker action 2021-02-15 18:29:27 -08:00
Shirshanka Das
76a9564a6f Adding github actions to build and push docker image 2021-02-15 18:29:27 -08:00
Shirshanka Das
8d74f97713 adding a file-to-file recipe 2021-02-15 18:29:27 -08:00
Shirshanka Das
327705fd0d MS-SQL integ test works 2021-02-15 18:29:27 -08:00
Shirshanka Das
d73d8ff4c2 fixing unit tests 2021-02-15 18:29:27 -08:00
Harshal Sheth
c7892ada4c Codegen avro + datahub kafka sink (#3) 
* Add codegen

* New architecture + setup file -> console pipeline

* Cleanup source loader

* Basic Kafka metadata source

* Kafka source and extractor

* Add kwargs construct interface

* Fix kafka source unit test

* start working on pipeline test

* kafka datahub sink

* Make myself a profile

* Ingest to datahub from kafka

* Update codegen

* Add restli transport

* Fix bug in restli conversion
 2021-02-15 18:29:27 -08:00
Shirshanka Das
b59a62fa1c setting modest coverage targets 2021-02-15 18:29:27 -08:00
Shirshanka Das
35e9f28b56 dropping unit tests 2021-02-15 18:29:27 -08:00
Shirshanka Das
60d861b498 dropping hard failures 2021-02-15 18:29:27 -08:00
Shirshanka Das
5e589da514 adding python action 2021-02-15 18:29:27 -08:00
Shirshanka Das
6b5bbbdc5f workaround for docker exec, waiting for 5 more seconds 2021-02-15 18:29:27 -08:00
Harshal Sheth
4fb673925c Start using avro producer 2021-02-15 18:29:27 -08:00
Shirshanka Das
9e61220132 checking in testing fixtures. docker still not working 2021-02-15 18:29:27 -08:00
Shirshanka Das
1ddbdee60c Support for SQL databases (MySQL + MS-SQL) (#2) 
* adding sql source + mysql

* adding sql support

* MSSQL support, basic integration test

* file sink and pipeline context
 2021-02-15 18:29:27 -08:00
Shirshanka Das
faf472aa64 adding some TODOs 2021-02-15 18:29:27 -08:00
Shirshanka Das
128781942d Firstdrop of ingest (#1) 2021-02-15 18:29:27 -08:00
Shirshanka Das
90b635fb7c Initial commit 2021-02-15 18:29:27 -08:00
Harshal Sheth
082c86463e Move old metadata ingestion scripts out of the way 2021-02-15 18:29:27 -08:00
John Joyce
54346b2e09 Fix build args 2021-02-15 17:40:41 -08:00
John Joyce
ef58e3ca43 rename 2021-02-15 17:40:41 -08:00
John Joyce
b6b13e88c2 Adding path 2021-02-15 17:40:41 -08:00
John Joyce
11f030b118 Adding Github Action to publish Docker image 2021-02-15 17:40:41 -08:00
John Joyce
98bfb9ac8e Dockerization 2021-02-15 17:40:41 -08:00
John Joyce
ef04205a73 Serving React App via Docker 2021-02-15 17:40:41 -08:00
John Joyce
e3ac44cfd4 Fixes 2021-02-15 17:40:41 -08:00
John Joyce
8ef5e2a545 Deploy React via Docker 2021-02-15 17:40:41 -08:00
John Joyce
da271e06a1
refactor(React): Renaming components/ dir to app/ (#2104) 2021-02-12 13:57:38 -08:00
John Joyce
9af0e56a34
docs(React): React README (#2103) 
* Adding React Documentation

* Add image registry

Co-authored-by: John Joyce <john@acryl.io>
 2021-02-12 10:48:40 -08:00
John Joyce
98b14b84e2
fix(React): Fix Dataset Ownership Role Update Bug (#2102) 
* Fix Ownership Update Bugs

Co-authored-by: John Joyce <john@acryl.io>
 2021-02-12 10:30:51 -08:00