mirror of
https://github.com/open-metadata/OpenMetadata.git
synced 2025-07-08 17:50:28 +00:00
e2578d6be3
* GitBook: [#177] Documentation Update - Airflow * GitBook: [#195] Removing Cron from databaseServices * GitBook: [#196] Added trino * GitBook: [#197] removed cron from config * GitBook: [#198] Added Redash Documentation * GitBook: [#199] Added Bigquery Usage Documentation * GitBook: [#200] Added page link for presto * GitBook: [#201] Added Local Docker documentation * GitBook: [#202] Added Documentation for Local Docker Setup * GitBook: [#203] Added Git Command to clone Openmetadata in docs * GitBook: [#207] links update * GitBook: [#208] Updating Airflow Documentation * GitBook: [#210] Adding Python installation package under Airflow Lineage config * GitBook: [#211] Change the links to 0.5..0 * GitBook: [#213] Move buried connectors page up * GitBook: [#214] Update to connectors page * GitBook: [#215] Removed sub-categories * GitBook: [#212] Adding Discovery tutorial * GitBook: [#220] Updated steps to H2s. * GitBook: [#230] Complex queries * GitBook: [#231] Add lineage to feature overview * GitBook: [#232] Make feature overview headers verbs instead of nouns * GitBook: [#233] Add data reliability to features overview * GitBook: [#234] Add complex data types to feature overview * GitBook: [#235] Simplify and further distinguish discovery feature headers * GitBook: [#236] Add data importance to feature overview * GitBook: [#237] Break Connectors into its own section * GitBook: [#238] Reorganize first section of docs. * GitBook: [#239] Add connectors to feature overview * GitBook: [#240] Organize layout of feature overview into feature categories as agreed with Harsha. * GitBook: [#242] Make overview paragraph more descriptive. * GitBook: [#243] Create a link to Connectors section from feature overview. * GitBook: [#244] Add "discover data through association" to feature overview. * GitBook: [#245] Update importance and owners gifs * GitBook: [#246] Include a little more descriptive documentation for key features. * GitBook: [#248] Small tweaks to intro paragraph. * GitBook: [#249] Clean up data profiler paragraph. * GitBook: [#250] Promote Complex Data Types to its own feature. * GitBook: [#251] Update to advanced search * GitBook: [#252] Update Roadmap * GitBook: [#254] Remove old features page (text and screenshot based). * GitBook: [#255] Remove references to removed page. * GitBook: [#256] Add Descriptions and Tags section to feature overview. * GitBook: [#257] Update title for "Know Your Data" Co-authored-by: Ayush Shah <ayush.shah@deuexsolutions.com> Co-authored-by: Suresh Srinivas <suresh@getcollate.io> Co-authored-by: Shannon Bradshaw <shannon.bradshaw@arrikto.com> Co-authored-by: OpenMetadata <github@harsha.io>
108 lines
3.9 KiB
Markdown
108 lines
3.9 KiB
Markdown
# Build the code & run tests
|
|
|
|
## Prerequisites
|
|
|
|
* Make sure you are using maven 3.5.x or higher and JDK 11 or higher.
|
|
* Make sure you have a local instance of MySQL and ElasticSearch.
|
|
* For an easy install of MySQL and ES, just install Docker on your local machine and run the following commands from the top-level directory
|
|
```
|
|
cd docker/local-metadata
|
|
docker-compose -f docker-compose-dev.yml up
|
|
```
|
|
* Bootstrap MySQL with tables
|
|
1. Create a distribution as explained [here](#create-a-distribution-packaging)
|
|
2. Extract the distribution tar.gz file and run the following command
|
|
```
|
|
cd open-metadata-<version>/bootstrap
|
|
sh bootstrap_storage.sh drop-create
|
|
```
|
|
* Bootstrap ES with indexes and load sample data into MySQL
|
|
1. Run OpenMetadata service instances through IntelliJ IDEA following the intstructions [here](#run-instance-through-intellij-idea)
|
|
2. Once the logs indicate that the instance is up, run the following commands from the top-level directory
|
|
```
|
|
python3 -m venv /tmp/venv
|
|
source /tmp/venv/bin/activate
|
|
pip install -r ingestion/requirements.txt
|
|
pip install -e ingestion
|
|
cd ingestion
|
|
metadata ingest -c ./pipelines/sample_data.json
|
|
metadata ingest -c ./pipelines/metadata_to_es.json
|
|
```
|
|
* You are now ready to explore the app by going to http://localhost:8585
|
|
*If the web page doesn't work as intended, pleasetake a look at the troubleshooting steps [here](#troubleshooting)
|
|
|
|
## Building
|
|
|
|
The following commands must be run from the top-level directory.
|
|
|
|
`mvn clean install`
|
|
|
|
If you wish to skip the unit tests you can do this by adding`-DskipTests`to the command line.
|
|
|
|
## Create a distribution (packaging)
|
|
|
|
You can create a _distribution_ as follows.
|
|
|
|
```
|
|
$ mvn clean install
|
|
|
|
# Create the binary distribution.
|
|
$ cd dist && mvn package
|
|
```
|
|
|
|
The binaries will be created at:
|
|
|
|
```
|
|
dist/target/open-metadata-<version>.pom
|
|
dist/target/open-metadata-<version>.tar.gz
|
|
```
|
|
|
|
## Run instance through IntelliJ IDEA
|
|
|
|
Add a new Run/Debug configuration like the below screenshot.
|
|
|
|
.png>)
|
|
|
|
## Add missing dependency
|
|
|
|
Right-click on catalog-rest-service
|
|
|
|
|
|
|
|
Click on "Open Module Settings"
|
|
|
|
|
|
|
|
Go to "Dependencies"
|
|
|
|
|
|
|
|
Click “+” at the bottom of the dialog box and click "Add"
|
|
|
|
|
|
|
|
Click on Library
|
|
|
|
|
|
|
|
In that list look for "jersey-client:2.25.1"
|
|
|
|
|
|
|
|
Select it and click "OK". Now run/debug the application.
|
|
|
|
## Troubleshooting
|
|
* If you see blank page at http://localhost:8585 , please check the logs at `logs/openmetadata.log`. You might be encountering one of the following errors:
|
|
* `connection refused` or `unreachable` - please confirm that MySQL and ES are reachable outside of docker by running `docker ps` and checking that ports 3306 and 9200 are listening on 0.0.0.0
|
|
* If ElasticSearch in Docker on Mac is crashing, try changing Preferences -> Resources -> Memory to 4GB
|
|
* If ElasticSearch logs show `high disk watermark [90%] exceeded`, try changing Preferences -> Resources -> Disk Image Size to at least 16GB
|
|
* `Public Key Retrieval is not allowed` - verify that the jdbc connect url in `conf/openmetadata.yaml` is configured with the parameter `allowPublicKeyRetrieval=true`
|
|
* Browser console shows javascript errors, try doing a [clean build](#building). Some npm packages may not have built properly.
|
|
|
|
## Coding Style
|
|
|
|
1. [Refer to coding guidelines](coding-style.md)
|
|
2. Configure IntelliJ to disable the \[wild-card imports]
|
|
|
|
([https://www.jetbrains.com/help/idea/creating-and-optimizing-imports.html#disable-wildcard-imports](https://www.jetbrains.com/help/idea/creating-and-optimizing-imports.html#disable-wildcard-imports))
|