refactor(react): React as the default UI (#2252)

Co-authored-by: John Joyce <john@acryl.io>
This commit is contained in:
John Joyce 2021-03-18 09:55:05 -07:00 committed by GitHub
parent b8462028c3
commit 728a742528
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
20 changed files with 97 additions and 86 deletions

View File

@ -78,6 +78,3 @@ jobs:
file: ./docker/datahub-frontend/Dockerfile
tags: ${{ steps.docker_meta.outputs.tags }}
push: ${{ needs.setup.outputs.publish == 'true' }}
build-args: |
SERVER_PORT=9002
ENABLE_REACT=true

View File

@ -78,3 +78,6 @@ jobs:
file: ./docker/datahub-frontend/Dockerfile
tags: ${{ steps.docker_meta.outputs.tags }}
push: ${{ needs.setup.outputs.publish == 'true' }}
build-args: |
SERVER_PORT=9001
ENABLE_EMBER=true

View File

@ -32,10 +32,11 @@ HOSTED_DOCS_ONLY-->
[![Version](https://img.shields.io/github/v/release/linkedin/datahub?include_prereleases)](https://github.com/linkedin/datahub/releases)
[![build & test](https://github.com/linkedin/datahub/workflows/build%20&%20test/badge.svg?branch=master&event=push)](https://github.com/linkedin/datahub/actions?query=workflow%3A%22build+%26+test%22+branch%3Amaster+event%3Apush)
[![Docker Pulls](https://img.shields.io/docker/pulls/linkedin/datahub-gms.svg)](https://hub.docker.com/r/linkedin/datahub-gms)
[![Get on Slack](https://img.shields.io/badge/slack-join-orange.svg)](https://join.slack.com/t/datahubspace/shared_invite/zt-dkzbxfck-dzNl96vBzB06pJpbRwP6RA)
[![Slack](https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack&style=social)](http://slack.datahubproject.io)
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://github.com/linkedin/datahub/blob/master/docs/CONTRIBUTING.md)
[![GitHub commit activity](https://img.shields.io/github/commit-activity/m/linkedin/datahub)](https://github.com/linkedin/datahub/pulls?q=is%3Apr)
[![License](https://img.shields.io/github/license/linkedin/datahub)](https://github.com/linkedin/datahub/blob/master/LICENSE)
[![Follow](https://img.shields.io/twitter/follow/datahubproject?label=Follow&style=social)](https://twitter.com/datahubproject)
---

View File

@ -18,11 +18,11 @@ However, if you only want to build `DataHub Frontend` specifically:
./gradlew :datahub-frontend:build
```
### Building React App
To build datahub-frontend to serve the React app, build with the additional "enableReact" property:
### Building Ember App
To build datahub-frontend to serve the legacy Ember app, build with the additional "enableEmber" property:
```
./gradlew :datahub-frontend:build -PenableReact=true
./gradlew :datahub-frontend:build -PenableEmber=true
```
## Dependencies
@ -58,9 +58,6 @@ the application directly from command line after a successful [build](#build):
cd datahub-frontend/run && ./run-local-frontend
```
### Serving React App
If you are running the React app locally via `yarn start`, it will be forwarding graphql requests to port `9002`. In order to use `./run-local-frontend` with the React app, change the PORT value in [./run/frontend.env](./run/frontend.env) to `9002` and restart `./run-local-frontend`
## Checking out DataHub UI
After starting your application in one of the two ways mentioned above, you can connect to it by typing below
into your favorite web browser:

View File

@ -13,10 +13,10 @@ configurations {
}
dependencies {
if (project.hasProperty('enableReact') && project.getProperty('enableReact').toBoolean()) {
assets project(path: ':datahub-web-react', configuration: 'assets')
} else {
if (project.hasProperty('enableEmber') && project.getProperty('enableEmber').toBoolean()) {
assets project(path: ':datahub-web', configuration: 'assets')
} else {
assets project(path: ':datahub-web-react', configuration: 'assets')
}
play project(":datahub-dao")

View File

@ -1,5 +1,5 @@
# Server Port
PORT=9001
PORT=9002
# Secret Key
DATAHUB_SECRET="YouKnowNothing"

View File

@ -1,22 +1,24 @@
# DataHub React App
## About
This module contains a React version of the DataHub UI, which is currently under incubation. Notice that this
is a completely separate frontend experience from Ember and will remain so as it evolves.
This module contains a React version of the DataHub UI. This is now the production version of the DataHub client experience.
Notice that this is a completely separate frontend experience from the legacy Ember app and will remain so as it evolves.
Feel free to take a look around, deploy, and make contributions.
Feel free to take a look around, deploy, and contribute.
For details about the motivation please see [this RFC](../docs/rfc/active/2055-react-app/README.md).
## Functional Goals
The initial milestone for the app is to achieve functional parity with the existing Ember app. This means supporting
The initial milestone for the app was to achieve functional parity with the existing Ember app. This meant supporting
- Dataset Profiles, Search, Browse Experience
- User Profiles, Search
- LDAP Authentication Flow
This has since been achieved. The new set of functional goals are reflected in the latest version of the [DataHub Roadmap](../docs/roadmap.md).
## Design Goals
In building out a new client experience, we intend to build on learnings from the Ember app and incorporate feedback gathered
In building out the client experience, we intend to leverage learnings from the Ember app and incorporate feedback gathered
from organizations operating DataHub. Two themes have emerged to serve as guideposts:
1. **Configurability**: The client experience should be configurable, such that deploying organizations can tailor certain
@ -33,7 +35,7 @@ from organizations operating DataHub. Two themes have emerged to serve as guidep
Navigate to the `docker` directory and run the following to spin up the react app:
```
./quickstart-react.sh
./quickstart.sh
```
at `http://localhost:9002`.
@ -71,14 +73,13 @@ you to terminate and re-run `yarn start` to see updated styles.
### Package Organization
The organization is yet subject to change as the app incubates. As of today (2/11/2021), the `src` dir of the app is
broken down into the following modules
The `src` dir of the app is broken down into the following modules
**conf** - Stores global configuration flags that can be referenced across the app. For example, the number of
search results shown per page, or the placeholder text in the search bar box. It serves as a location where levels
for functional configurability should reside.
**components** - Contains all important components of the app. It has a few sub-modules:
**app** - Contains all important components of the app. It has a few sub-modules:
- `auth`: Components used to render the user authentication experience.
- `browse`: Shared components used to render the 'browse-by-path' experience. The experience is akin to navigating a filesystem hierarchy.

View File

@ -1,8 +1,14 @@
DataHub Web Client
DataHub Ember Client (Legacy)
==============================================================================
>**Notice**: As of March 2021, the React application will be the default frontend experience supported by the community.
> Going forward, major feature development will occur on the React application. Eventually, support for Ember will be
> deprecated altogether (tentatively mid-2021).
>
> Please migrate to the React app, which resides under the `datahub-web-react` directory.
## About
This repository is for the portal web-client and related packages for DataHub, LinkedIn's premier data search and
This repository is for the legacy web-client and related packages for DataHub, LinkedIn's premier data search and
discovery tool. It is written as an Ember application in a mono-repository. For more information about our code
architecture choices and logic, please visit the `documentation` folder [here](documentation/MAIN.md).
@ -59,8 +65,8 @@ ember g datahub-addon
### Quick Start
First, follow the instructions on the DataHub Quickstart Guide, which will run an instance of the backend for DataHub
and also serve the static docker image from `http://localhost:9001`. This is the best way to see the current latest
First, follow the instructions on the DataHub Quickstart Guide for Ember (`quickstart-ember.sh`), which will
run an instance of the backend for DataHub and also serve the static docker image from `http://localhost:9001`. This is the best way to see the current latest
frontend/UI in action as well. However, if you want to make changes and see them live without having to rebuild a
docker image, you can, from this `datahub-web` folder run the following:

View File

@ -24,7 +24,7 @@ You can easily download and run all these images and their dependencies with our
DataHub Docker Images:
* [linkedin/datahub-gms](https://cloud.docker.com/repository/docker/linkedin/datahub-gms/)
* [linkedin/datahub-frontend](https://cloud.docker.com/repository/docker/linkedin/datahub-frontend/)
* [linkedin/datahub-frontend-react](https://cloud.docker.com/repository/docker/linkedin/datahub-frontend-react/)
* [linkedin/datahub-mae-consumer](https://cloud.docker.com/repository/docker/linkedin/datahub-mae-consumer/)
* [linkedin/datahub-mce-consumer](https://cloud.docker.com/repository/docker/linkedin/datahub-mce-consumer/)
@ -61,31 +61,31 @@ COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose -p datahub build
This is because we're relying on builtkit for multistage builds. It does not hurt also set `DATAHUB_VERSION` to
something unique.
## React
To serve the incubating React UI, follow the instructions below.
## Ember
To serve the legacy Ember UI, follow the instructions below.
> **Before continuing**: If you've already run a deploy script, don't forget to clear containers using `docker container prune`
### Serving React Only
### Serving Ember Only
#### All Containers
Use the `quickstart-react.sh` script to launch all containers in DataHub, including a frontend server that returns a React UI
Use the `quickstart-ember.sh` script to launch all containers in DataHub, including a frontend server that serves the Ember UI
```
./quickstart-react.sh
./quickstart-ember.sh
```
#### The Bare Minimum
Run the following command to launch only the React server and its required dependencies
Run the following command to launch only the Ember server and its required dependencies
```
docker-compose -f docker-compose.react.yml -f docker-compose.yml -f docker-compose.override.yml up datahub-frontend-react
docker-compose -f docker-compose.ember.yml -f docker-compose.yml -f docker-compose.override.yml up datahub-frontend-ember
```
Once complete, navigate to `localhost:9002/` in your browser to see the React app.
Once complete, navigate to `localhost:9001` in your browser to see the legacy Ember app.
### Serving React + Ember
If you'd like to serve the React and Ember UIs side-by-side, you can deploy the `datahub-frontend-react` container manually.
If you'd like to serve the React and Ember UIs side-by-side, you can deploy the `datahub-frontend-ember` service manually.
#### All Containers
@ -94,22 +94,22 @@ To deploy all DataHub containers, run the quickstart script:
./quickstart.sh
```
Next, deploy the container that serves the React UI:
Next, deploy the container that serves the Ember UI:
```
docker-compose -f docker-compose.react.yml -f docker-compose.yml -f docker-compose.override.yml up --no-deps datahub-frontend-react
docker-compose -f docker-compose.ember.yml -f docker-compose.yml -f docker-compose.override.yml up --no-deps datahub-frontend-ember
```
#### The Bare Minimum
First, start the Ember frontend server & its required dependencies:
First, start the React frontend server & its required dependencies:
```
docker-compose up datahub-frontend
docker-compose up datahub-frontend-react
```
Then, start the React frontend server & its required dependencies:
Then, start the Ember frontend server & its required dependencies:
```
docker-compose -f docker-compose.react.yml -f docker-compose.yml -f docker-compose.override.yml up datahub-frontend-react
docker-compose -f docker-compose.ember.yml -f docker-compose.yml -f docker-compose.override.yml up datahub-frontend-ember
```
Navigate to `localhost:9001/` to view the Ember app & `localhost:9002/` to view the React app.
Navigate to `localhost:9002/` to view the React app & `localhost:9001/` to view the legacy Ember app.

View File

@ -1,6 +1,6 @@
FROM openjdk:8 as builder
ARG ENABLE_REACT="false"
ARG ENABLE_EMBER="false"
RUN apt-get update && apt-get install -y wget \
&& wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb \
@ -9,7 +9,7 @@ RUN apt-get update && apt-get install -y wget \
ENV CI=true
COPY . datahub-src
RUN cd datahub-src && ./gradlew :datahub-frontend:dist -PenableReact=${ENABLE_REACT} \
RUN cd datahub-src && ./gradlew :datahub-frontend:dist -PenableEmber=${ENABLE_EMBER} \
&& cp datahub-frontend/build/distributions/datahub-frontend.zip ../datahub-frontend.zip \
&& cd .. && rm -rf datahub-src && unzip datahub-frontend.zip
@ -21,7 +21,7 @@ COPY --from=builder /datahub-frontend /datahub-frontend/
RUN chown -R datahub:datahub /datahub-frontend && chmod 755 /datahub-frontend
USER datahub
ARG SERVER_PORT=9001
ARG SERVER_PORT=9002
RUN echo $SERVER_PORT
EXPOSE $SERVER_PORT

View File

@ -9,6 +9,12 @@ responsibility of this service for the DataHub.
After starting your Docker container, you can connect to it by typing below into your favorite web browser:
If using React app:
```
http://localhost:9002
```
If using legacy Ember app:
```
http://localhost:9001
```

View File

@ -0,0 +1,19 @@
# Introduces datahub-frontend-ember service to serve the Ember UI on localhost:9001/
---
version: '3.8'
services:
datahub-frontend-ember:
build:
context: ../
dockerfile: docker/datahub-frontend/Dockerfile
args:
SERVER_PORT: 9001
ENABLE_EMBER: "true"
image: linkedin/datahub-frontend:${DATAHUB_VERSION:-latest}
env_file: datahub-frontend/env/docker.env
hostname: datahub-frontend
container_name: datahub-frontend
ports:
- "9001:9001"
depends_on:
- datahub-gms

View File

@ -1,19 +0,0 @@
# Introduces datahub-frontend-react container to serve the React UI on localhost:9002/
---
version: '3.8'
services:
datahub-frontend-react:
build:
context: ../
dockerfile: docker/datahub-frontend/Dockerfile
args:
SERVER_PORT: 9002
ENABLE_REACT: "true"
image: linkedin/datahub-frontend-react:${DATAHUB_VERSION:-latest}
env_file: datahub-frontend/env/docker.env
hostname: datahub-frontend-react
container_name: datahub-frontend-react
ports:
- "9002:9002"
depends_on:
- datahub-gms

View File

@ -143,16 +143,16 @@ services:
- mysql
- neo4j
datahub-frontend:
datahub-frontend-react:
build:
context: ../
dockerfile: docker/datahub-frontend/Dockerfile
image: linkedin/datahub-frontend:${DATAHUB_VERSION:-latest}
image: linkedin/datahub-frontend-react:${DATAHUB_VERSION:-latest}
env_file: datahub-frontend/env/docker.env
hostname: datahub-frontend
container_name: datahub-frontend
hostname: datahub-frontend-react
container_name: datahub-frontend-react
ports:
- "9001:9001"
- "9002:9002"
depends_on:
- datahub-gms

View File

@ -1,10 +1,10 @@
#!/bin/bash
# Quickstarts a React-serving variant of DataHub by pulling all images from dockerhub and then running the containers locally.
# Quickstarts a Ember-serving variant of DataHub by pulling all images from dockerhub and then running the containers locally.
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
cd $DIR && docker-compose -f docker-compose.yml -f docker-compose.override.yml -f docker-compose.react.yml pull && docker-compose -p datahub \
cd $DIR && docker-compose -f docker-compose.yml -f docker-compose.override.yml -f docker-compose.ember.yml pull && docker-compose -p datahub \
-f docker-compose.yml \
-f docker-compose.override.yml \
-f docker-compose.react.yml \
-f docker-compose.ember.yml \
up \
--scale datahub-frontend=0
--scale datahub-frontend-react=0

View File

@ -1,6 +1,6 @@
#!/bin/bash
# Quickstarts DataHub by pullinng all images from dockerhub and then running the containers locally. No images are
# Quickstarts DataHub by pulling all images from dockerhub and then running the containers locally. No images are
# built locally. Note: by default this pulls the latest version; you can change this to a specific version by setting
# the DATAHUB_VERSION environment variable.
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

View File

@ -6,7 +6,7 @@ You can list all Docker containers in your local by running `docker container ls
```
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
979830a342ce linkedin/datahub-mce-consumer:latest "bash -c 'while ping…" 10 hours ago Up 10 hours datahub-mce-consumer
3abfc72e205d linkedin/datahub-frontend:latest "datahub-frontend/bi…" 10 hours ago Up 10 hours 0.0.0.0:9001->9001/tcp datahub-frontend
3abfc72e205d linkedin/datahub-frontend-react:latest "datahub-frontend…" 10 hours ago Up 10 hours 0.0.0.0:9002->9002/tcp datahub-frontend
50b2308a8efd linkedin/datahub-mae-consumer:latest "bash -c 'while ping…" 10 hours ago Up 10 hours datahub-mae-consumer
4d6b03d77113 linkedin/datahub-gms:latest "bash -c 'dockerize …" 10 hours ago Up 10 hours 0.0.0.0:8080->8080/tcp datahub-gms
c267c287a235 landoop/schema-registry-ui:latest "/run.sh" 10 hours ago Up 10 hours 0.0.0.0:8000->8000/tcp schema-registry-ui
@ -24,9 +24,9 @@ Also you can check individual Docker container logs by running `docker logs <<co
2020-02-06 09:20:54.870:INFO:oejs.Server:main: Started @18807ms
```
For `datahub-frontend`, you should see a log similar to this at the end of the initialization:
For `datahub-frontend-react`, you should see a log similar to this at the end of the initialization:
```
09:20:22 [main] INFO play.core.server.AkkaHttpServer - Listening for HTTP on /0.0.0.0:9001
09:20:22 [main] INFO play.core.server.AkkaHttpServer - Listening for HTTP on /0.0.0.0:9002
```
## My elasticsearch or broker container exited with error or was stuck forever

View File

@ -16,7 +16,7 @@ DataHub can be right for you if you want an open source unbundled solution (fron
Currently LinkedIn engineers. However, were receiving more and more PRs from individuals working at various companies.
## How big is the community?
We had couple of meetings/discussions with external parties who are interested in DataHub, for example:
External parties who deploy / are actively evaluating DataHub:
- [City of New York, DoITT](https://www1.nyc.gov/site/doitt/index.page)
- [Experian](https://www.experian.com/)
- [Geotab](https://www.geotab.com/)

View File

@ -24,6 +24,11 @@ Our open sourcing [blog post](https://engineering.linkedin.com/blog/2020/open-so
- **Browse**: browsing through a configurable hierarchy [*coming soon*]
- **Profile editing**: LinkedIn style professional profile editing such as summary, skills
### Dashboards
- **Search**: full-text & advanced search, search ranking
- **Basic information**: ownership, location. Link to external service for viewing the dashboard.
- **Institutional knowledge**: support for adding free form doc to any dashboards [*coming soon*]
### Schemas [*coming soon*]
- **Search**: full-text & advanced search, search ranking
- **Browse**: browsing through a configurable hierarchy
@ -41,11 +46,6 @@ Our open sourcing [blog post](https://engineering.linkedin.com/blog/2020/open-so
- **Browse**: browsing through a configurable hierarchy
- **Basic information**: ownershp, dimensions, formula, input & output datasets, dashboards
- **Institutional knowledge**: support for adding free form doc to any metric
### Dashboards [*coming soon*]
- **Search**: full-text & advanced search, search ranking
- **Basic information**: ownership, location. Link to exzternal service for viewing the dashboard.
- **Institutional knowledge**: support for adding free form doc to any dashboards
## Metadata Sources
There's a basic, Java-oriented overview of [metadata ingestion](architecture/metadata-ingestion.md).

View File

@ -8,7 +8,7 @@
./docker/quickstart.sh
```
This step takes a while to run the first time, and it may be difficult to tell if DataHub is fully up and running from the combined log. Please use [this guide](debugging.md#how-can-i-confirm-if-all-docker-containers-are-running-as-expected-after-a-quickstart) to verify that each container is running correctly.
5. At this point, you should be able to start DataHub by opening [http://localhost:9001](http://localhost:9001) in your browser. You can sign in using `datahub` as username and any password (no password validation by default). However, you'll notice that no data has been ingested yet.
5. At this point, you should be able to start DataHub by opening [http://localhost:9002](http://localhost:9002) in your browser. You can sign in using `datahub` as username and any password (no password validation by default). However, you'll notice that no data has been ingested yet.
6. To ingest provided [sample data](https://github.com/linkedin/datahub/blob/master/metadata-ingestion/mce-cli/bootstrap_mce.dat) to DataHub, switch to a new terminal window, `cd` into the cloned `datahub` repo, and run the following command:
```
./docker/ingestion/ingestion.sh