82 Commits

Author SHA1 Message Date
Swaroop Jagadish
df0b4f258b
feat(ingest): support for primary and foreign key extraction from sql sources (#3316) 2021-09-30 23:04:47 -07:00
Gabe Lyons
ff527f4bed
feat(foreign keys): add foreign key models (#3275) 2021-09-22 10:29:27 -07:00
rslanka
8844240328
feat: Adding support for nested schemas in ingestion and visualization (#3079) 2021-08-11 15:47:18 -07:00
John Joyce
352a0abf8d
Introducing TimeSeries Aspects + Dataset Profile (Stats) Aspect (#2983)
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-07-30 17:41:03 -07:00
Gabe Lyons
aa253f5b3b
feat(deletes): add run commands (list, show, rollback) to datahub ingest (#2960) 2021-07-29 20:04:40 -07:00
Harshal Sheth
be39037b10
build(ingest): reduce dependencies for dev install (#2872) 2021-07-14 20:02:48 -07:00
Harshal Sheth
2f921d15e8
fix(ingest): avoid setting timestamps unless source system provides it (#2843) 2021-07-08 12:11:06 -07:00
Harshal Sheth
d66381451a
feat(ingest): refactor mce comparison and add pytest update golden files option (#2812) 2021-06-30 16:53:20 -07:00
Kevin Hu
554e1637c5
fix(ingest): types for dbt (#2716) 2021-06-22 10:37:08 -07:00
Harshal Sheth
31eae24300
fix(ingest): support mssql encryption via ODBC (#2657) 2021-06-04 18:19:11 -07:00
shakti-garg
8ed14a62e2
feat(business_glossary): add new entity business term and its relationship with dataset and its fields (#2228)
Co-authored-by: shubham.garg <shubham.garg@thoughtworks.com>
2021-05-10 13:20:23 -07:00
Harshal Sheth
a11329d5b8
refactor(ingest): update test harness to use a compose file per test (#2392) 2021-04-13 17:30:24 -07:00
Harshal Sheth
cb24628886
feat(ingest): verify dynamic registry types at runtime (#2327) 2021-04-01 12:15:05 -07:00
Harshal Sheth
aa6bc15cd7
fix(ingest): various avro codegen fixes (#2232) 2021-03-15 15:27:30 -07:00
Harshal Sheth
4e15feebfd
fix(ingestion): nullable types and timestamp precision (#2212) 2021-03-10 23:29:24 -08:00
Gabe Lyons
adfe60e97a
feat(tags): adding support for read/write of tags in gms & read-only in react datahub-frontend. (#2164) 2021-03-07 11:26:47 -08:00
Harshal Sheth
0310e97b5b
feat(ingest): capture table descriptions (#2157) 2021-03-02 11:49:37 -08:00
Harshal Sheth
9336cbc743
test(ingest): verify the output of mssql (#2120) 2021-02-18 14:47:49 -08:00
Harshal Sheth
0063c04460 gometa-ingest -> datahub ingest 2021-02-15 18:29:27 -08:00
Harshal Sheth
d0bc3c55db Setup CI 2021-02-15 18:29:27 -08:00
Harshal Sheth
f807bb44b8 Use type + config everywhere 2021-02-15 18:29:27 -08:00
Harshal Sheth
df3e3da45b More autofixes 2021-02-15 18:29:27 -08:00
Harshal Sheth
7ca018aaa4 Remove unused imports 2021-02-15 18:29:27 -08:00
Harshal Sheth
43d5fac494 Black 2021-02-15 18:29:27 -08:00
Shirshanka Das
7ba81d7087 remove old shell scripts 2021-02-15 18:29:27 -08:00
Harshal Sheth
a87161cad7 Run black formatting on tests 2021-02-15 18:29:27 -08:00
Harshal Sheth
36874970ce Finish file sink 2021-02-15 18:29:27 -08:00
Harshal Sheth
23d675663b Minor test runid renames 2021-02-15 18:29:27 -08:00
Shirshanka Das
4e17a24dd4 Adding mysql integration test + source 2021-02-15 18:29:27 -08:00
Harshal Sheth
4b83fc6591 adding allow deny patterns to sql config 2021-02-15 18:29:27 -08:00
Shirshanka Das
6b5bbbdc5f workaround for docker exec, waiting for 5 more seconds 2021-02-15 18:29:27 -08:00
Shirshanka Das
9e61220132 checking in testing fixtures. docker still not working 2021-02-15 18:29:27 -08:00