3975 Commits

Author SHA1 Message Date
John Joyce
8b716ad2b4
Adding SampleHiveDataset bootstrap metadata (#3947) 2022-01-21 14:39:51 -08:00
Aseem Bansal
6c3441ccdc
fix(docs): fix openapi docs (#3940) 2022-01-21 07:55:50 -08:00
Aseem Bansal
f3d01009df
docs(ingest): fix command for running quick tests (#3942) 2022-01-21 07:40:31 -08:00
RyanHolstien
85ff0f3630
fix(ingest): telemetry - soft fail (#3934) 2022-01-21 07:35:42 -08:00
Aseem Bansal
4aa14214d9
feat(ingest): rest-emitter: make test_connection more robust (#3919)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-01-19 18:51:47 -08:00
Aseem Bansal
22efdec2ab
Revert "fix(glossary): owner should be optional as per docs (#3858)" (#3910)
This reverts commit d59574f671f8599933f5b6072d2a65d5e4f3b341.
2022-01-18 09:33:05 -08:00
Kevin Hu
d4075b972c
docs(ingest): data lake - describe path-spec configuration (#3905) 2022-01-17 15:41:02 -08:00
Hoang-Vu Le
d5ef7cefa2
fix(ingest): nifi - replace hardcode password with config variable (#3902)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-01-17 15:36:37 -08:00
Kevin Hu
9e30b42638
feat(ingest): data lake - support for profiling data lake assets (#3656) 2022-01-17 15:01:23 -08:00
Jiwei
f804184884
docs(elasticsearch): fix typo in config example (#3901) 2022-01-17 14:56:17 -08:00
Aseem Bansal
400e0fe838
feat(ingest): kafka - support schema references (#3862) 2022-01-17 14:29:54 -08:00
Ravindra Lanka
1efe04f88a
feat(ingest): glue - support for nested structs (#3895) 2022-01-17 14:21:53 -08:00
Aseem Bansal
d59574f671
fix(glossary): owner should be optional as per docs (#3858) 2022-01-17 08:40:16 -08:00
Swaroop Jagadish
7d986ec880
fix(ingest): populate system metadata for all metadata events (mcp, mcpw) (#3900) 2022-01-16 12:03:38 -08:00
Ravindra Lanka
a44b48a6b8
feat(ingest): elasticsearch - add Elasticsearch Source (#3893)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-01-14 13:10:12 -08:00
Tamas Nemeth
2daa06a58f
fix(ingest): redshift - mapping for spectrum external array types (#3890) 2022-01-14 12:23:57 -08:00
Tamas Nemeth
e95446be1c
fix(ingest): sqlparser - Not lowercasing looker source's special table name (#3891)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-01-14 12:22:17 -08:00
varunbharill
cc52e3e759
fix(ingest): bigquery - update dependencies for module. (#3874)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-01-13 22:12:11 -08:00
iasoon
de50a34201
feat(ingest): dbt - support spark sql types (#3880) 2022-01-13 12:37:29 -08:00
iasoon
285aa1b9ac
docs(ingest): azure-ad - specify required permission type (#3886) 2022-01-13 12:30:16 -08:00
Kevin Hu
88b1e8c426
feat(ingest): ingestion-specific telemetry (#3881) 2022-01-12 21:47:42 -08:00
Aseem Bansal
7498f4bf54
doc(kafka): add permissions required for confluent cloud (#3850) 2022-01-12 14:50:58 -08:00
iasoon
988baeb467
feat(ingest): metabase - allow configuring how database engines get mapped to platforms (#3869) 2022-01-12 00:52:55 -08:00
Kevin Hu
5f701ebb3c
fix(cli): disable telemetry in CLI tests (#3877) 2022-01-12 00:25:42 -08:00
iasoon
130d5ff635
docs(ingest): azure-ad - correct default value for username attr (#3861) 2022-01-11 19:37:28 -08:00
Rickard Cardell
2f7e49b399
feat(ingest): datahub-rest - Make datahub-rest client more robust by configurable retries. (#3826) (#3860) 2022-01-11 19:35:06 -08:00
iasoon
6f7c2120fd
fix(ingest): metabase - fix crashes on missing values (#3859) 2022-01-10 15:40:52 -08:00
Harshal Sheth
0171735ae0
feat(build): include correct version in metadata-ingestion docker image (#3857) 2022-01-10 15:38:19 -08:00
Aditya Radhakrishnan
c3e98645a2
feat(ingest): operational stats - show last updated for sql usage sources (#3845) 2022-01-07 17:27:09 -08:00
iasoon
0a6ec819cf
fix(ingest): dbt - handle database =None for executors like spark (#3851) 2022-01-07 15:50:29 -08:00
Paolo Genissel
744dee5206
fix(ingest): bigquery-usage - fix remove_extras to remove all partitions (#3842) 2022-01-07 12:30:45 -08:00
Aseem Bansal
cfe3c54d02
fix(ingest): dbt - fix warning due to struct type not being mapped (#3846) 2022-01-07 12:15:36 -08:00
Aseem Bansal
c4e401ee48
fix(azure AD): fix problem with missing key causing failures in ingestion (#3824) 2022-01-07 06:22:49 -08:00
Ravindra Lanka
9f80e5487d
fix(ingest): metabase - fix the datetime parsing issue (#3831)
fixes #3803
2022-01-05 16:34:05 -08:00
Swaroop Jagadish
480ad24282
feat(ingest): dbt - add support for dbt meta mapping (#3832) 2022-01-05 16:32:05 -08:00
Kevin Hu
5b369447f4
fix(ingest): glue - ignore custom connectors (#3805) 2022-01-05 15:12:50 -08:00
Tamas Nemeth
0f20d60bfc
feat(profiler): using approximate queries for profiling (#3752) 2022-01-05 12:42:15 -08:00
Tamas Nemeth
e5d8313788
feat(ingest): sql parsing - Replacing sqlmetadata sql parser lib with sqlineage parser lib (#3806) 2022-01-05 12:22:59 -08:00
Tamas Nemeth
286b42663c
fix(ingest): redshift - handle string type for Glue-backed tables (#3811) 2022-01-04 01:43:41 -08:00
Ravindra Lanka
81bc173630
fix(ingest): Make url an optional field of the DefaultConfig for business glossary source. (#3817) 2022-01-04 01:42:01 -08:00
MugdhaHardikar-GSLab
c65609a84a
feat(datahub-client): add Java REST emitter (#3781) 2022-01-02 22:48:38 +05:30
Tamas Nemeth
450cdc1739
feat(snowflake-usage): Generate email address if not exists (#3791) 2021-12-29 08:14:18 -08:00
Aseem Bansal
895af09af2
docs(kafka): add example for ingestion from confluent cloud (#3789) 2021-12-28 07:17:03 -08:00
Tamas Nemeth
5df5150e51
feat(ingest): Add config option to set Bigquery credential in source config (#3786) 2021-12-27 05:48:45 -08:00
Sergio Gómez Villamor
0f8458ad74
doc(transforms): adds doc for simple_add_dataset_properties transformer (#3790) 2021-12-27 05:46:09 -08:00
cccs-eric
2645881b08
fix(superset): fix Superset chart ingestion with an empty metric label (#3792) (#3793) 2021-12-27 05:44:37 -08:00
Tamas Nemeth
c2459056e2
docs(redshift): Updating Redshift permission prerequisites in doc (#3777) 2021-12-22 14:15:09 -08:00
Sergio Gómez Villamor
3fa22e1138
feat(transform): adds simple add dataset properties transform (#3778) 2021-12-22 08:51:38 -08:00
Hyun Min Choi
e76d40b57a
fix(lkml): bump lkml version up to 1.1.2 to support sql_preamble expression (#3757) 2021-12-21 12:13:13 -08:00
Aseem Bansal
77e3641a50
fix(ingest): fix compatibility with google composer (#3774) 2021-12-20 12:47:56 -08:00