yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-06-27 05:03:31 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(ingest): update metadata-ingestion dev guide (#12779)

Browse Source
This commit is contained in:
Harshal Sheth 2025-03-07 11:22:17 -08:00 committed by GitHub
parent a51713a378
commit 8ff905f0d3
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 5 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
metadata-ingestion/developing.md
View File

@ -155,8 +155,8 @@ Instead, we recommend using UI-based ingestion or isolating the ingestion pipeli
The syntax for installing plugins is slightly different in development. For example:
```diff
- pip install 'acryl-datahub[bigquery,datahub-rest]'
+ pip install -e '.[bigquery,datahub-rest]'
- uv pip install 'acryl-datahub[bigquery,datahub-rest]'
+ uv pip install -e '.[bigquery,datahub-rest]'
```
## Architecture
@ -180,7 +180,7 @@ The architecture of this metadata ingestion framework is heavily inspired by [Ap
We use ruff, and mypy to ensure consistent code style and quality.
```shell
# Assumes: pip install -e '.[dev]' and venv is activated
# Assumes: ../gradlew :metadata-ingestion:installDev and venv is activated
ruff check src/ tests/
mypy src/ tests/
```
@ -247,11 +247,8 @@ In order to ensure that the configs are consistent and easy to use, we have a fe
```shell
# Follow standard install from source procedure - see above.
# Install, including all dev requirements.
pip install -e '.[dev]'
# For running integration tests, you can use
pip install -e '.[integration-tests]'
# Install all dev and test requirements.
../gradlew :metadata-ingestion:installDevTest
# Run the full testing suite
pytest -vv