yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-12-27 18:07:57 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: update local dev guide (#12585)

Browse Source 
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
This commit is contained in:
Hyejin Yoon 2025-02-14 12:56:09 +09:00 committed by GitHub
parent 10a7e01f4f
commit 5ba7767a85
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 51 additions and 13 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

64
docs/developers.md
View File

@ -77,38 +77,76 @@ We suggest partially compiling DataHub according to your needs:
./gradlew :docs-website:serve
```
## Deploying Local Versions
## Deploying Local Versions
This guide explains how to set up and deploy DataHub locally for development purposes.
Run just once to have the local `datahub` cli tool installed in your $PATH
### Initial Setup
Before you begin, you'll need to install the local `datahub` CLI tool:
```shell
cd smoke-test/
cd metadata-ingestion/
python3 -m venv venv
source venv/bin/activate
pip install --upgrade pip wheel setuptools
pip install -r requirements.txt
cd ../
```
Once you have compiled & packaged the project or appropriate module you can deploy the entire system via docker-compose by running:
### Deploying the Full Stack
Deploy the entire system using docker-compose:
```shell
./gradlew quickstart
./gradlew quickstartDebug
```
Replace whatever container you want in the existing deployment.
I.e, replacing datahub's backend (GMS):
Access the DataHub UI at `http://localhost:9002`
### Refreshing the Frontend
To run and update the frontend with local changes, open a new terminal and run:
```shell
(cd docker && COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker compose -p datahub -f docker-compose-without-neo4j.yml -f docker-compose-without-neo4j.override.yml -f docker-compose.dev.yml up -d --no-deps --force-recreate --build datahub-gms)
cd datahub-web-react
yarn install && yarn start
```
The frontend will be available at `http://localhost:3000` and will automatically update as you make changes to the code.
### Refreshing GMS
To refresh the GMS (Generalized Metadata Service) with local changes:
```shell
./gradlew :metadata-service:war:build -x test --parallel && docker restart datahub-datahub-gms-debug-1
```
Running the local version of the frontend
### Refreshing the CLI
```shell
(cd docker && COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker compose -p datahub -f docker-compose-without-neo4j.yml -f docker-compose-without-neo4j.override.yml -f docker-compose.dev.yml up -d --no-deps --force-recreate --build datahub-frontend-react)
If you haven't set up the CLI for local development yet, run:
```commandline
./gradlew :metadata-ingestion:installDev
cd metadata-ingestion
source venv/bin/activate
```
Once you're in `venv`, your local changes will be reflected automatically.
For example, you can run `datahub ingest -c <file>` to test local changes in ingestion connectors.
To verify that you're using the local version, run:
```commandline
datahub --version
```
Expected Output:
```commandline
acryl-datahub, version unavailable (installed in develop mode)
```
### Refreshing Other Components
To refresh other components with local changes, just run:
```commandline
./gradlew quickstartDebug
```
## IDE Support
The recommended IDE for DataHub development is [IntelliJ IDEA](https://www.jetbrains.com/idea/).