11918 Commits

Author SHA1 Message Date
Kerem Sahin
06fae51ca3 Fix NPE issue with downstream lineage resource 2019-09-26 20:53:38 -07:00
Kerem Sahin
53b79de9bf Fix the issue with validating array types 2019-09-26 20:52:30 -07:00
Kerem Sahin
40478e2d26 Adding support for surfacing downstream dataset relationships using search index until we have graph ready 2019-09-26 11:59:11 -07:00
Kerem Sahin
1ec31bbfb7 Start indexing upstream datasets to search index to support downstream dataset relationships 2019-09-26 11:57:58 -07:00
Chris Lee
646630b599 Enabled the kafka ingestion pipeline. 2019-09-24 16:41:06 -07:00
Chris Lee
49534ea590 Aligned the long type in time format. v0.1.0-alpha 2019-09-19 09:48:43 -07:00
Seyi Adebajo
6e158b2fd5
Merge pull request #1502 from theseyi/datahub
Fixes login tests related to token removal
2019-09-18 17:22:24 -07:00
Seyi Adebajo
59ed313bcd Fixes login tests related to token removal 2019-09-18 17:19:59 -07:00
Seyi Adebajo
81e34e8d22
Merge pull request #1501 from theseyi/datahub
Sets flag to show token credential input to false
2019-09-18 12:56:31 -07:00
Seyi Adebajo
6080ddfae9 Sets flag to show token credential input to false 2019-09-18 12:55:59 -07:00
Seyi Adebajo
dfd69b0f8a
Merge pull request #1500 from theseyi/datahub
Updates the tests and autocomplete facets to origin string
2019-09-18 12:46:09 -07:00
Seyi Adebajo
829a10ce09 Updates the tests and autocomplete facets to origin string 2019-09-18 12:45:31 -07:00
Seyi Adebajo
33a493e0c3
Merge pull request #1499 from theseyi/datahub
Updates the origin facet to origin instead of datatorigin
2019-09-18 12:14:28 -07:00
Seyi Adebajo
424e7f79cb Updates the origin facet to origin instead of datatorigin 2019-09-18 11:56:22 -07:00
Seyi Adebajo
e482add6b2
Merge pull request #1498 from theseyi/datahub
Resolves related ticket issues and misc tasks readying for OS
2019-09-18 11:19:34 -07:00
Chris Lee
600c2f04ab Organized the output layout. 2019-09-18 11:03:48 -07:00
Seyi Adebajo
f66f81dc4c META-9929, META-9868, resolves related ticket issues and misc tasks 2019-09-18 10:07:04 -07:00
Kerem Sahin
162d52a421 Updated DataHub wiki link conf 2019-09-17 16:30:37 -07:00
Chris Lee
72b18aa890 Introduced the hive ETL pipeline. 2019-09-17 16:14:09 -07:00
Chris Lee
57976cc901 Introduced the hive ETL pipeline. 2019-09-17 16:13:35 -07:00
Chris Lee
9cc7a1e2f8 Imported the confluent-kafka[avro] as a requirement. 2019-09-17 15:06:56 -07:00
Chris Lee
e6b98c1ba1 Merge branch 'datahub' of https://github.com/linkedin/WhereHows into datahub 2019-09-17 15:04:02 -07:00
Chris Lee
5c2e03b162 Imported the confluent-kafka[avro] as a requirement. 2019-09-17 15:03:25 -07:00
Seyi Adebajo
c57d7719a6
Updates default pictureLink in CorpUserEditableInfo
Updated to path https://raw.githubusercontent.com/linkedin/WhereHows/datahub/datahub-web/packages/data-portal/public/assets/images/default_avatar.png
2019-09-17 08:50:47 -07:00
Kerem Sahin
339c9bea86 Update default pictureLink 2019-09-16 15:55:11 -07:00
Chris Lee
493cbe16d6 Added the runbook for the requirements.txt. 2019-09-16 13:27:16 -07:00
Chris Lee
bd32ec20a1 Supplied the requirements.txt for metadata-ingestion. 2019-09-16 13:13:42 -07:00
Chris Lee
81a38f87f7 Classified the ETL jobs in metadata-ingestion. 2019-09-16 12:52:37 -07:00
Kerem Sahin
28b876f323 Update docs: No need for running initialization script for creating search indices 2019-09-12 19:06:35 -07:00
Kerem Sahin
1a3ddff4a4 Creating an image for elasticsearch-setup to automatically create indices 2019-09-12 19:04:29 -07:00
Chris Lee
430f13108d Enabled the configuration lists in the LDAP ETL. 2019-09-12 10:16:41 -07:00
Chris Lee
9df9d336c9 Ingested metadata from LDAP server to Data Hub. 2019-09-10 18:58:22 -07:00
Kerem Sahin
1bf0ff72b4 Doc update: adding docker-compose pull before up to always get the latest version of images 2019-09-10 17:51:17 -07:00
Kerem Sahin
c365c406e1 Normalize browse paths for datasets 2019-09-10 15:14:40 -07:00
Kerem Sahin
6716e38279 Update documentation 2019-09-09 01:45:49 -07:00
Kerem Sahin
8bfb086e09 Set displayName fields for corp user MCEs 2019-09-09 00:31:09 -07:00
Kerem Sahin
822dc22d8e Add pictureLink to DatasetOwner view 2019-09-08 23:59:25 -07:00
Kerem Sahin
fcd1b46f41 Add dataset urn to dataset client API value model 2019-09-08 23:02:44 -07:00
Kerem Sahin
0dc5bd9fe0 Add documentation 2019-09-08 20:25:58 -07:00
Kerem Sahin
693ee97c6c Fix ingestion script by pointing to correct MCE schema
Refactor for metadata-ingestion module
Adding readme for metadata-ingestion
2019-09-08 05:27:19 -07:00
Kerem Sahin
e3e6dc1d30 Remove kafka rest-proxy container which is not required 2019-09-08 01:10:38 -07:00
Kerem Sahin
9f43305fe9 Add mce-consumer to quickstart docker-compose 2019-09-08 00:57:04 -07:00
Kerem Sahin
b82cb7df69 Add dependency to kafka-setup container for mae-consumer job 2019-09-08 00:56:31 -07:00
Kerem Sahin
fea754c5be Implement docker build & docker-compose for mce-consumer job 2019-09-08 00:54:03 -07:00
Kerem Sahin
e9c557f0e3 Start reading configuration from environment variables for mce-consumer 2019-09-08 00:07:02 -07:00
Kerem Sahin
3a5e34d744 Add mae-consumer to quickstart docker-compose 2019-09-07 23:44:41 -07:00
Kerem Sahin
9090ac48d1 Implement docker build & docker-compose for mae-consumer job 2019-09-07 23:43:33 -07:00
Kerem Sahin
958c376354 Bump confluent kafka images to 5.3.0 and create another container for Kafka topics initialization 2019-09-07 23:41:59 -07:00
Kerem Sahin
2584793cb5 Start reading configuration from environment variables for mae-consumer 2019-09-07 23:38:56 -07:00
Kerem Sahin
cc93351b63 Bump kafka-clients & kafka-streams to 2.3.0 2019-09-07 23:35:23 -07:00