mirror of
https://github.com/open-metadata/OpenMetadata.git
synced 2025-12-11 15:28:00 +00:00
Update setup, ingestion docs
This commit is contained in:
parent
df6030fe12
commit
40911af11f
@ -37,8 +37,7 @@
|
||||
|
||||
## Install
|
||||
|
||||
* [Run Locally](install/run-locally.md)
|
||||
* [Run Docker](install/run-docker.md)
|
||||
* [Run OpenMetadata](install/run-openmetadata.md)
|
||||
* [Run in production](install/run-in-production.md)
|
||||
* [Metadata Ingestion](install/metadata-ingestion/README.md)
|
||||
* [Ingest Sample Data](install/metadata-ingestion/ingest-sample-data.md)
|
||||
|
||||
@ -1,2 +1,21 @@
|
||||
# Metadata Ingestion
|
||||
|
||||
OpenMetadata Ingesiton is a simple framework to build connectors and ingest metadata of various systems through OpenMetadata APIs.
|
||||
It could be used in an orchestration framework(e.g. Apache Airflow) to ingest metadata.
|
||||
|
||||
## Ingest Sample Data
|
||||
{% page-ref page="ingest-sample-metadata" %}
|
||||
|
||||
|
||||
## Explore and configure connectors
|
||||
{% page-ref page="connectors" %}
|
||||
|
||||
|
||||
## Run Simple Scheduler
|
||||
|
||||
{% page-ref page="scheduler" %}
|
||||
|
||||
|
||||
## Run using Airflow
|
||||
|
||||
{% page-ref page="airflow" %}
|
||||
|
||||
@ -1,2 +1,58 @@
|
||||
---
|
||||
description: >-
|
||||
This installation doc will help you start a OpenMetadata instances
|
||||
on your production.
|
||||
---
|
||||
|
||||
|
||||
# Run in production
|
||||
|
||||
Please refer to the previous section [Run Openmetadata](install/run-openmetadata.md) for configuring OpenMetadata.
|
||||
|
||||
{% hint style="info" %}
|
||||
**Prerequisites**
|
||||
|
||||
* MySQL >= 8.x
|
||||
* ElasticSearch >= 7.x
|
||||
* Airflow or other schedulers to run Ingestion Connectors
|
||||
|
||||
{% endhint %}
|
||||
|
||||
|
||||
## Start OpenMetadata
|
||||
|
||||
OpenMetadata release ships with ```./bin/openmetadata``` init.d style
|
||||
script.
|
||||
|
||||
```text
|
||||
cd openmetdata-0.3.0
|
||||
./bin/openmetdata start
|
||||
```
|
||||
|
||||
We recommend to configure serviced to monitor openmetadata command to restart
|
||||
incase of any failures.
|
||||
|
||||
|
||||
## Running with a load balancer
|
||||
|
||||
One or more OpenMetadata instances can be put behind a load balancer for reverse proxying,
|
||||
in that case appropriate OpenMetdata url must be mentioned in the load balancer's configuraiton file.
|
||||
|
||||
For example, in case Apache mod proxy the VirtualHost tag in the configuration file should be
|
||||
edited out with the following
|
||||
|
||||
```text
|
||||
|
||||
<VirtualHost *:80>
|
||||
<Proxy balancer://mycluster>
|
||||
BalancerMember http://127.0.0.1:8585 <!-- First OpenMetadata server -->
|
||||
BalancerMember http://127.0.0.2:8686 <!-- Second OpenMetadata server -->
|
||||
</Proxy>
|
||||
|
||||
ProxyPreserveHost On
|
||||
|
||||
ProxyPass / balancer://mycluster/
|
||||
ProxyPassReverse / balancer://mycluster/
|
||||
</VirtualHost>
|
||||
```
|
||||
|
||||
@ -4,6 +4,27 @@ description: >-
|
||||
on your local machine.
|
||||
---
|
||||
|
||||
# Run Docker
|
||||
|
||||
[Docker](https://docs.docker.com/get-started/overview/) is an open platform for developing, shipping, and running applications that enables you to separate your applications from your infrastructure so you can deliver software quickly using OS-level virtualization to deliver software in packages called containers.
|
||||
|
||||
{% hint style="info" %}
|
||||
**Prerequisites**
|
||||
|
||||
* Docker >= 20.10.x
|
||||
|
||||
**Ports to access once the docker is up:**
|
||||
|
||||
* OpenMetadata UI: 8585
|
||||
* Scheduler UI: 7777
|
||||
{% endhint %}
|
||||
|
||||
```text
|
||||
cd docker/metadata
|
||||
docker-compose up
|
||||
```
|
||||
|
||||
|
||||
# Run Locally
|
||||
|
||||
{% hint style="success" %}
|
||||
@ -22,6 +43,18 @@ OpenMetadata is built using Java, DropWizard, Jetty, and MySQL.
|
||||
{% endhint %}
|
||||
|
||||
{% tabs %}
|
||||
{% tab title="Download the release" %}
|
||||
Download the latest binary release from [OpenMetadata](https://open-metadata.org/download/), Once you have the tar file,
|
||||
|
||||
```bash
|
||||
# untar it
|
||||
tar -zxvf openmetadata-0.3.0.tar.gz
|
||||
|
||||
# navigate to directory containing the launcher scripts
|
||||
cd openmetadata-0.3.0
|
||||
```
|
||||
{% endtab %}
|
||||
|
||||
{% tab title="Build from source " %}
|
||||
Follow these steps to checkout code from [Github](https://github.com/open-metadata/OpenMetadata) and build OpenMetadata locally
|
||||
|
||||
@ -46,17 +79,7 @@ cd openmetadata-1.0.0-SNAPSHOT
|
||||
```
|
||||
{% endtab %}
|
||||
|
||||
{% tab title="Download the release" %}
|
||||
Download the latest binary release from [OpenMetadata](https://open-metadata.org/download/), Once you have the tar file,
|
||||
|
||||
```bash
|
||||
# untar it
|
||||
tar -zxvf openmetadata-1.0.0-SNAPSHOT.tar.gz
|
||||
|
||||
# navigate to directory containing the launcher scripts
|
||||
cd openmetadata-1.0.0-SNAPSHOT
|
||||
```
|
||||
{% endtab %}
|
||||
{% endtabs %}
|
||||
|
||||
## Install on your local machine
|
||||
@ -91,13 +114,14 @@ cd openmetadata-1.0.0-SNAPSHOT
|
||||
2. Run bootstrap scripts to initiate the database and tables
|
||||
|
||||
```text
|
||||
cd $METADATA_HOME
|
||||
cd openmetadata-0.3.0
|
||||
./boostrap/bootstrap-storage.sh migrate
|
||||
```
|
||||
|
||||
3. Start the OpenMetadata Server
|
||||
|
||||
```text
|
||||
cd openmetadata-0.3.0
|
||||
./bin/openmetadata.sh start
|
||||
```
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user