Harshal Sheth
|
d415234a8c
|
fix(ingest): fields with defaults should be optional (#2461)
|
2021-04-26 16:45:48 -07:00 |
|
Harshal Sheth
|
2da5e1fd10
|
feat(ingest): setup scaffolding for tox testing (#2451)
|
2021-04-26 16:44:36 -07:00 |
|
Harshal Sheth
|
034c33a050
|
fix(ingest): use entrypoints lib instead of pkg_resources (#2438)
|
2021-04-22 00:13:47 -07:00 |
|
Gabe Lyons
|
c7b49de67b
|
feat(ingest): adding superset ingestion source (#2425)
|
2021-04-22 00:11:54 -07:00 |
|
Harshal Sheth
|
ffe49f061a
|
fix(ingest): fix chart type enum serialization and add tests for rest emitter (#2429)
|
2021-04-21 11:34:24 -07:00 |
|
Harshal Sheth
|
79daec29b7
|
fix(ingest): ensure upstreams in airflow lineage emission are entities (#2427)
|
2021-04-20 20:44:38 -07:00 |
|
Harshal Sheth
|
9ac17c4ee0
|
fix(ingest): bump avro-gen3 (#2403)
Closes #2375.
|
2021-04-16 11:59:05 -07:00 |
|
Harshal Sheth
|
777c05973f
|
fix(ingest): add sqlalchemy extra (#2409)
|
2021-04-16 09:41:23 -07:00 |
|
Harshal Sheth
|
ffe03e6758
|
fix(ingest): streamline codegen init methods (#2400)
|
2021-04-14 19:25:57 -07:00 |
|
Harshal Sheth
|
fb6f74b1da
|
feat(ingest): add generic sqlalchemy source (#2389)
|
2021-04-13 08:01:38 -07:00 |
|
Harshal Sheth
|
41cd52f9e2
|
feat(ingest): add Airflow lineage backend (#2368)
|
2021-04-12 17:40:15 -07:00 |
|
Harshal Sheth
|
b0d8f70354
|
fix(ingest): bump pybigquery version (#2352)
|
2021-04-06 18:34:06 -07:00 |
|
Harshal Sheth
|
cfc02ee196
|
feat(ingest): add Oracle db support (#2347)
|
2021-04-06 15:38:25 -07:00 |
|
Harshal Sheth
|
bd78b84bd3
|
feat(ingest): start airflow integration + metadata builders (#2331)
|
2021-04-05 19:11:28 -07:00 |
|
Harshal Sheth
|
c1f3eaed35
|
fix(ingest): add support for database and table patterns to glue source (#2339)
|
2021-04-05 17:14:02 -07:00 |
|
amy m
|
759288161c
|
feat(ingest): adding support for AWS Glue (#2319)
Co-authored-by: Harry Nash <harrywilliamnash@gmail.com>
|
2021-04-04 11:00:27 -07:00 |
|
Harshal Sheth
|
cb24628886
|
feat(ingest): verify dynamic registry types at runtime (#2327)
|
2021-04-01 12:15:05 -07:00 |
|
Joe Mirizio
|
f3304bec7c
|
feat(ingest): dynamically register plugins (#2316)
Co-authored-by: Joe Mirizio <mirizioj@email.chop.edu>
|
2021-03-31 20:59:45 -07:00 |
|
Harshal Sheth
|
f57c954fc6
|
feat(ingest): support environment variables in recipes (#2306)
|
2021-03-26 21:57:05 -07:00 |
|
Harshal Sheth
|
07f4cb1199
|
feat: datahub check local-docker (#2295)
|
2021-03-26 10:03:51 -07:00 |
|
Harshal Sheth
|
cc19465f55
|
fix(ingest): resolve array serialization bug (#2290)
|
2021-03-24 10:02:46 -07:00 |
|
Harshal Sheth
|
a921d0deae
|
feat(ingest): MongoDB ingestion source (#2289)
|
2021-03-23 20:15:44 -07:00 |
|
Harshal Sheth
|
1ea450e0e4
|
fix(ingest): use custom pybigquery ref to get descriptions (#2279)
|
2021-03-22 23:17:54 -07:00 |
|
Harshal Sheth
|
b8462028c3
|
feat(ingest): various minor fixes (#2246)
|
2021-03-17 23:05:05 -07:00 |
|
Pedro Silva
|
6a0c402a58
|
feat(ingest): Add support for druid (#2235)
|
2021-03-17 20:06:48 -07:00 |
|
Harshal Sheth
|
aa6bc15cd7
|
fix(ingest): various avro codegen fixes (#2232)
|
2021-03-15 15:27:30 -07:00 |
|
Harshal Sheth
|
95c124ffc4
|
fix(ingest): pin version of avro-gen3 (#2230)
|
2021-03-12 09:39:38 -08:00 |
|
Harshal Sheth
|
6a8fca59f1
|
feat(ingest): use plugin system based on Python extras (#2224)
|
2021-03-11 13:41:05 -08:00 |
|
Harshal Sheth
|
dced25fef7
|
feat(ingest): switch quickstart to Python ingestion (#2158)
|
2021-03-02 11:48:26 -08:00 |
|
Harshal Sheth
|
347148b79b
|
Update python workflow
|
2021-02-15 18:29:27 -08:00 |
|
Harshal Sheth
|
38f75be8ad
|
gometa -> datahub
|
2021-02-15 18:29:27 -08:00 |
|
Harshal Sheth
|
0063c04460
|
gometa-ingest -> datahub ingest
|
2021-02-15 18:29:27 -08:00 |
|
Harshal Sheth
|
b91d0cf63b
|
Add bigquery and refactor others
|
2021-02-15 18:29:27 -08:00 |
|
Harshal Sheth
|
d12497a3ff
|
Add postgres source
|
2021-02-15 18:29:27 -08:00 |
|
Harshal Sheth
|
3678368ed3
|
Prune deps
|
2021-02-15 18:29:27 -08:00 |
|
Harshal Sheth
|
9332e6b878
|
Add isort to CI
|
2021-02-15 18:29:27 -08:00 |
|
Harshal Sheth
|
0660991fb8
|
More python 3.6 compat
|
2021-02-15 18:29:27 -08:00 |
|
Harshal Sheth
|
75b01806ef
|
Use find_packages() in setup
|
2021-02-15 18:29:27 -08:00 |
|
Shirshanka Das
|
9c6ea3b9e9
|
Adding 3.9 to setup.py classifiers
|
2021-02-15 18:29:27 -08:00 |
|
Shirshanka Das
|
024745bcc4
|
removing py 3.6, adding 3.9 for ci
|
2021-02-15 18:29:27 -08:00 |
|
Harshal Sheth
|
c7892ada4c
|
Codegen avro + datahub kafka sink (#3)
* Add codegen
* New architecture + setup file -> console pipeline
* Cleanup source loader
* Basic Kafka metadata source
* Kafka source and extractor
* Add kwargs construct interface
* Fix kafka source unit test
* start working on pipeline test
* kafka datahub sink
* Make myself a profile
* Ingest to datahub from kafka
* Update codegen
* Add restli transport
* Fix bug in restli conversion
|
2021-02-15 18:29:27 -08:00 |
|
Harshal Sheth
|
4fb673925c
|
Start using avro producer
|
2021-02-15 18:29:27 -08:00 |
|
Shirshanka Das
|
1ddbdee60c
|
Support for SQL databases (MySQL + MS-SQL) (#2)
* adding sql source + mysql
* adding sql support
* MSSQL support, basic integration test
* file sink and pipeline context
|
2021-02-15 18:29:27 -08:00 |
|
Shirshanka Das
|
faf472aa64
|
adding some TODOs
|
2021-02-15 18:29:27 -08:00 |
|
Shirshanka Das
|
128781942d
|
Firstdrop of ingest (#1)
|
2021-02-15 18:29:27 -08:00 |
|