mirror of https://github.com/datahub-project/datahub.git synced 2025-11-02 03:39:03 +00:00

Mars Lan 6b7609918e Replace sbt build with native Gradle Play plugin and update the docs. (#352 )

Benefits
1. Simpler setup - no need to download activator in order to build & run
2. Faster build - See https://engineering.linkedin.com/play/developing-play-applications-using-gradle
3. Streamlined dependency management - Everything defined in build.gradle, instead of build.gradle + build.sbt
4. Better integration with gradle lifecycle tasks - build, test, dist, clean all work as expected

Changes
1. Location of staging & distribution files moved from target to build
2. Use ./gradle -t runPlayBinary to run app with hot reload support
3. The generated start scripts are quite different from those generated by sbt

2017-07-10 09:54:08 -07:00

3.2 KiB

Raw Blame History

WhereHows

WhereHows is a data discovery and lineage tool built at LinkedIn. It integrates with all the major data processing systems and collects both catalog and operational metadata from them.

Within the central metadata repository, WhereHows curates, associates, and surfaces the metadata information through two interfaces:

a web application that enables data & linage discovery, and community collaboration
an API endpoint that empowers automation of data processes/applications

WhereHows serves as the single platform that:

links data objects with people and processes
enables crowdsourcing for data knowledge
provides data governance and provenance based on ownership and lineage

Documentation

The detailed information can be found in the Wiki

Examples in VM

There is a pre-built vmware image (about 11GB) to quickly demonstrate the functionality of WhereHows. Check out the VM Guide

Getting Started

New to Wherehows? Check out the Getting Started Guide

Preparation

First, please setup the metadata repository in MySQL.

CREATE DATABASE wherehows
  DEFAULT CHARACTER SET utf8
  DEFAULT COLLATE utf8_general_ci;

CREATE USER 'wherehows';
SET PASSWORD FOR 'wherehows' = PASSWORD('wherehows');
GRANT ALL ON wherehows.* TO 'wherehows'

Execute the DDL files to create the required repository tables in wherehows database.

Build

Get the source code: git clone https://github.com/linkedin/WhereHows.git
Put a few 3rd-party jar files to metadata-etl/extralibs directory. Some of these jar files may not be available in Maven Central or Artifactory. See the download instrucitons for more detail. cd WhereHows/metadata-etl/extralibs
From the WhereHows root directory and build all the modules: ./gradlew build
Start the metadata ETL and API service: ./gradlew backend-service:runPlayBinary
In a new terminal, start the web front-end: ./gradlew web:runPlayBinary. The WhereHows UI is available at http://localhost:9000 by default. You can change the port number by editing the value of project.ext.httpPort in web/build.gradle.

Contribute

Want to contribute? Check out the Contributors Guide

Community

Want help? Check out the Gitter chat room and Google Groups

3.2 KiB Raw Blame History