3089 Commits

Author SHA1 Message Date
taufiqibrahim
dd1b81b727
feat(ingest): adding redash source (#2918)
Co-authored-by: tibrahim <taufiq.ibrahim@bizzy.co.id>
2021-08-18 12:03:03 -07:00
John Joyce
f40bf1ce19
feat(group ui): Basic group search membership in UI (#3094) 2021-08-16 20:47:18 -07:00
Shikha-Trivedi-Saxo
c9c1ba457e
feat(business-glossary): Glossary term relationship backend (#3012) 2021-08-15 23:05:03 -07:00
Shirshanka Das
39a0081800
fix(ingest): increasing default ingestion REST timeout to 30 seconds (#3095) 2021-08-13 11:05:44 -07:00
Gabe Lyons
b73d0ebb8b
feat: Adding clarity around qualified unions and removing extra lines for structs (#3091)
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2021-08-12 16:25:44 -07:00
John Joyce
8c9c696cdd
feat(ingest): Adding an Okta Integration to extract Users, Groups, Group Membership (#3043) 2021-08-11 18:49:16 -07:00
rslanka
8844240328
feat: Adding support for nested schemas in ingestion and visualization (#3079) 2021-08-11 15:47:18 -07:00
Chinmay Bhat
7abcb45f8c
feat(ingest): add support for jdbc connector to kafka-connect source (#3076) 2021-08-11 08:50:38 -07:00
Rahul Jain
0e0198f385
feat(ingest): Added support for "add dataset ownership by regex match" (#2990) 2021-08-10 11:03:16 -07:00
Kevin Hu
5891482114
fix(ingest): apply case insensitive regex matching by default (#3067) 2021-08-10 10:35:57 -07:00
Gabe Lyons
4e15c7adbb
feat(cli): datahub init & docs for it (#3062) 2021-08-09 22:30:48 -07:00
aseembansal-gogo
9ee091f518
feat(ingest): add transformers to clear dataset ownership, mark status, add browse paths (#2986) 2021-08-09 09:43:46 -07:00
Dexter Lee
0956b79b4d
fix(ingest): stop looker source from unnecessarily filling out owners (#3056) 2021-08-08 23:07:05 -07:00
Dexter Lee
8316316daa
fix(ingest): add default configurable timeout for rest emitter (#3055) 2021-08-08 22:30:55 -07:00
Gabe Lyons
fef2ea7126
feat(quickstart): linking to slack from quickstart (#3051) 2021-08-06 13:03:59 -07:00
Kevin Hu
40ecd96c7a
feat(ingest): replace and warn against relative imports (#3033) 2021-08-06 10:25:30 -07:00
aseembansal-gogo
ac345e0ade
feat(ingest): add underlying platform for glue (#3035) 2021-08-06 10:19:21 -07:00
Kevin Hu
3d0534be4c
fix(ingest): detect malformed Glue S3 script paths (#3037) 2021-08-05 20:06:32 -07:00
Kevin Hu
f1bea875b2
fix(ingest): replace backticks for lookml (#3038) 2021-08-05 20:05:20 -07:00
Kevin Hu
ec3dfebc24
fix(ingest): add bigquery type mappings (#3040) 2021-08-05 20:03:21 -07:00
Kevin Hu
3d061161d8
fix(ingest): glue import type stubs only for testing (#3032) 2021-08-04 12:43:22 -07:00
Kevin Hu
3701931363
feat(ingest): extract dbt versions into custom properties (#3021) 2021-08-03 12:13:49 -07:00
Kevin Hu
07bbd50ad2
fix(ingest): don't assume Glue job description always exists (#3019) 2021-08-03 11:42:43 -07:00
Dexter Lee
51a2669332
feat(ingest): add kafka emitters for MetadataChangeProposal format (#3018) 2021-08-03 11:17:16 -07:00
James Lamb
fd205221b2
fix(ingest): map all LookML dimension types to corresponding avro types (#2972) 2021-08-02 11:09:05 -07:00
Dexter Lee
283f6376d1
fix(ingestion): Safeguard against empty values for profile ingestion (#3005) 2021-08-02 10:42:40 -07:00
John Joyce
352a0abf8d
Introducing TimeSeries Aspects + Dataset Profile (Stats) Aspect (#2983)
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-07-30 17:41:03 -07:00
Gabe Lyons
aa253f5b3b
feat(deletes): add run commands (list, show, rollback) to datahub ingest (#2960) 2021-07-29 20:04:40 -07:00
James Lamb
07ff99fdbe
fix(ingest): add more debug logging to LookML metadata ingestion (#2967)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-07-28 23:46:01 -07:00
James Lamb
acffd8fb4d
fix(ingest): ensure that LookML files are always parsed in the same order (#2966) 2021-07-28 23:20:49 -07:00
James Lamb
e88ccd93e3
fix(ingest): ensure workunits are created for all LookML views (#2965) 2021-07-28 23:13:15 -07:00
Kevin Hu
bf2775f6cf
feat(ingest): type stubs for boto3 (#2975) 2021-07-28 20:35:35 -07:00
James Lamb
b2dc4b96eb
perf(ingest): remove unused variable in Looker ingestion (#2979) 2021-07-28 20:33:27 -07:00
Harshal Sheth
1c334bb0e0
fix(ingest): infer bigquery project identifier (#2980)
Also refactor by moving get_identifier method into the source rather than leaving it in the config.
2021-07-28 20:31:33 -07:00
Chinmay Bhat
a33770b022
fix(ingest): fix hive ingestion to respect database configuration (#2978) 2021-07-28 20:20:18 -07:00
Harshal Sheth
7ab6355b1c
feat(ingest): stricter deserialization for MCE JSONs (#2976) 2021-07-28 14:50:21 -07:00
Harshal Sheth
14bca4c2f6
fix(ingestion): isolate dependency requirements of airflow hooks (#2977) 2021-07-28 14:23:06 -07:00
Harshal Sheth
c32ba3c5ef
feat(ingest): add timezone validation to bigquery usage (#2962) 2021-07-28 14:16:31 -07:00
Harshal Sheth
fa22b8e17b
feat(ingest): add Airflow TaskFlow example (#2958) 2021-07-26 13:09:25 -07:00
Kevin Hu
662017ef17
fix(ingest): patch lookml types and refactor ingestion sources layout (#2950) 2021-07-26 13:06:52 -07:00
James Lamb
5d396b1487
fix(ingest): match nested LookML files mentioned in 'include' statements (#2957) 2021-07-26 12:38:49 -07:00
Kevin Hu
f82ea3abef
feat(ingest): ingest descriptions from dbt models (#2955) 2021-07-26 10:47:25 -07:00
Harshal Sheth
62d5306a28
fix(ingestion): add more mypy annotations (#2948) 2021-07-25 21:19:25 -07:00
Harshal Sheth
f08cf11a4d
feat(ingestion): test GMS connections before ingestion (#2946) 2021-07-25 21:17:51 -07:00
Harshal Sheth
7a1ce19d2b
fix(ingestion): resolve test bugs for 3.6 (#2953) 2021-07-23 17:07:13 -07:00
Harshal Sheth
4bcfe92df0
fix(ingest): handle quotes in lookml properly (#2940) 2021-07-22 22:19:04 -07:00
Kevin Hu
43e61d9628
feat(models): remove versions from metrics and hyperparams (#2938) 2021-07-22 22:18:45 -07:00
Harshal Sheth
01982310be
feat(ingest): use urn builders in looker and validate data platforms (#2939) 2021-07-22 21:50:44 -07:00
aseembansal-gogo
d39030dab2
refactor(ingest): make code pythonic (#2941) 2021-07-22 21:47:08 -07:00
Kevin Hu
51bd2c6a81
fix(ingest): allow custom Glue scripts (#2937) 2021-07-22 21:45:48 -07:00