3904 Commits

Author SHA1 Message Date
Harshal Sheth
07f4cb1199
feat: datahub check local-docker (#2295) 2021-03-26 10:03:51 -07:00
Harshal Sheth
cc19465f55
fix(ingest): resolve array serialization bug (#2290) 2021-03-24 10:02:46 -07:00
Pedro Silva
ca2a7fb527
fix(ingest): Add druid-specific identification logic (#2284) 2021-03-23 20:25:05 -07:00
Harshal Sheth
a921d0deae
feat(ingest): MongoDB ingestion source (#2289) 2021-03-23 20:15:44 -07:00
Harshal Sheth
1ea450e0e4
fix(ingest): use custom pybigquery ref to get descriptions (#2279) 2021-03-22 23:17:54 -07:00
Harshal Sheth
c42ce2e668
docs(ingest): add python versions badge (#2278) 2021-03-22 23:12:41 -07:00
Gary Lucas
922307f8ec
feat(ingest): Add DBT source to metadata-ingestion (#2247) 2021-03-22 23:11:29 -07:00
Dexter Lee
61c37aa04d
refactor(demo): Add empty global tags to BigQuery demo data (#2258) 2021-03-18 16:03:05 -07:00
Gabe Lyons
039fe597f7
feat(tags): editing tags from react client on datasets, schemas, charts & dashboards (#2248) 2021-03-18 11:52:14 -07:00
Harshal Sheth
b8462028c3
feat(ingest): various minor fixes (#2246) 2021-03-17 23:05:05 -07:00
Pedro Silva
6a0c402a58
feat(ingest): Add support for druid (#2235) 2021-03-17 20:06:48 -07:00
Harshal Sheth
aa6bc15cd7
fix(ingest): various avro codegen fixes (#2232) 2021-03-15 15:27:30 -07:00
Arun Vasudevan
7750c6120a
feat: MLmodel Graphql Query (#2166) 2021-03-13 08:34:48 -08:00
Harshal Sheth
95c124ffc4
fix(ingest): pin version of avro-gen3 (#2230) 2021-03-12 09:39:38 -08:00
Harshal Sheth
6a8fca59f1
feat(ingest): use plugin system based on Python extras (#2224) 2021-03-11 13:41:05 -08:00
Thomas Larsson
53be2078a9
refactor(ingest): improve athena source api and documentation (#2219)
Co-authored-by: thomas.larsson <thomas.larsson@klarna.com>
2021-03-11 11:26:29 -08:00
Thomas Larsson
2b470c2d85
feat(ingest): add aws athena ingestion source (#2213)
Co-authored-by: thomas.larsson <thomas.larsson@klarna.com>
2021-03-11 10:12:27 -08:00
Thomas Larsson
1f1518ce10
feat(ingest): add option for optimized skipping of schemas (#2209)
Co-authored-by: thomas.larsson <thomas.larsson@klarna.com>
2021-03-11 00:08:01 -08:00
Harshal Sheth
4e15feebfd
fix(ingestion): nullable types and timestamp precision (#2212) 2021-03-10 23:29:24 -08:00
Harshal Sheth
6378347f81
feat(ingest): standalone metadata emitters (#2207) 2021-03-10 14:32:12 -08:00
Harshal Sheth
d220647094
feat: add date and time types to SQL model (#2201) 2021-03-09 23:07:20 -08:00
Gabe Lyons
0046aae7cf
feat(tags): Enriching sample data for tags (#2194) 2021-03-08 17:26:41 -08:00
Gabe Lyons
adfe60e97a
feat(tags): adding support for read/write of tags in gms & read-only in react datahub-frontend. (#2164) 2021-03-07 11:26:47 -08:00
Harshal Sheth
1f082b114b
feat(ingest): bigquery sample data (#2178) 2021-03-05 16:39:34 -08:00
Harshal Sheth
0d374f30c5
docs(ingest): clarify setuptools requirement (#2177) 2021-03-05 12:18:13 -08:00
Harshal Sheth
20bf794ec4
docs: hosted documentation website (#2174) 2021-03-05 00:12:12 -08:00
Harshal Sheth
ae9ffde8d0
fix(ingest): loosen Kafka broker validation (#2172) 2021-03-04 23:17:36 -08:00
Harshal Sheth
ec9c739306
docs(ingest): clarify Kafka connection config (#2171) 2021-03-04 11:42:57 -08:00
Harshal Sheth
e066991f54
fix(ingest): bigquery source and dataset naming fixes (#2161) 2021-03-03 19:49:46 -08:00
Harshal Sheth
0310e97b5b
feat(ingest): capture table descriptions (#2157) 2021-03-02 11:49:37 -08:00
Harshal Sheth
dced25fef7
feat(ingest): switch quickstart to Python ingestion (#2158) 2021-03-02 11:48:26 -08:00
Pedro Silva
6c7668115f
feat(ingest): support alternative authentication in sql ingestion (#2156) 2021-03-02 11:47:47 -08:00
Harshal Sheth
11532a1cc3
build(docker): add large generated directories to dockerignore (#2151) 2021-03-02 10:48:55 -08:00
Harshal Sheth
9e73794022
ci(ingest): setup docker container for metadata ingestion (#2150) 2021-03-01 17:36:38 -08:00
Harshal Sheth
656d9c771b
feat(ingest): support Postgres PostGIS extensions (#2136) 2021-02-25 09:05:16 -08:00
Harshal Sheth
d2745ee9cd
ci(ingest): run apt update (#2135) 2021-02-23 20:51:23 -08:00
Harshal Sheth
ac064584ae
refactor(ingest): cleanup configuration models (#2134) 2021-02-23 15:55:31 -08:00
Harshal Sheth
2a184dd3d5
fix(ingest): gracefully handle unknown types (#2125) 2021-02-19 19:57:13 -08:00
Harshal Sheth
e096c61a32
fix(ingest): update sample MCEs based on MLModel changes (#2124) 2021-02-19 17:03:27 -08:00
Harshal Sheth
b2fc27dbe8
docs(ingest): add solutions for common install issues (#2123) 2021-02-18 20:06:30 -08:00
Harshal Sheth
76e0594b8b
feat(ingest): add support for LDAP ingestion (#2122) 2021-02-18 20:05:39 -08:00
Harshal Sheth
9336cbc743
test(ingest): verify the output of mssql (#2120) 2021-02-18 14:47:49 -08:00
Harshal Sheth
dfe00bfee8
feat(ingest): improve error reporting for pipelines (#2121) 2021-02-18 11:15:13 -08:00
Harshal Sheth
02ffa6fd54
docs(ingest): add example DAGs for Airflow (#2116) 2021-02-17 18:01:22 -08:00
RyanHolstien
ea86ade29b
feat: ML Model Backend Implementation (#1896)
Co-authored-by: RyanHolstien <rholstien@expediagroup.com>
2021-02-17 13:28:13 -08:00
Harshal Sheth
fd4878a89d
docs(ingest): clarify docs for new ingestion framework (#2108) 2021-02-16 15:31:13 -08:00
Harshal Sheth
6e0c7ee37f Add architecture diagram 2021-02-15 18:29:27 -08:00
Harshal Sheth
6b750fc4fb Update docs 2021-02-15 18:29:27 -08:00
Harshal Sheth
347148b79b Update python workflow 2021-02-15 18:29:27 -08:00
Harshal Sheth
2d901a087a Remove docker scripts 2021-02-15 18:29:27 -08:00