mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-08-31 12:39:01 +00:00

Suresh Srinivas c4a89a2cab Update ingestion connector docs

2021-08-12 14:19:14 -07:00

1.2 KiB

Raw Blame History

description
This guide will help you to ingest sample data

Ingest Sample Data

Sample Data

{% hint style="info" %} Prerequisites

OpenMetadata is built using Java, DropWizard, Jetty, and MySQL.

Python 3.7 or above
Create and activate python env

{% endhint %}

Install from PyPI or Source

{% tabs %} {% tab title="Install Using PyPI" %}

pip install 'openmetadata-ingestion[sample-tables, elasticsearch]'

{% endtab %} {% tab title="Build from source " %}

# checkout OpenMetadata
git clone https://github.com/open-metadata/OpenMetadata.git
cd OpenMetadata/ingestion
python3 -m venv env
source env/bin/activate
pip install '.[sample-tables, elasticsearch]'

{% endtab %} {% endtabs %}

Ingest sample tables and users

metadata ingest -c ./pipelines/sample_tables.json
metadata ingest -c ./pipelines/sample_users.json

Index Sample Data into ElasticSearch

Start Elastic Search Docker:

docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.10.2

Index sample data in ElasticSearch:

metadata ingest -c ./pipelines/metadata_to_es.json

1.2 KiB Raw Blame History