mirror of
				https://github.com/open-metadata/OpenMetadata.git
				synced 2025-10-31 10:39:30 +00:00 
			
		
		
		
	
		
			
				
	
	
		
			85 lines
		
	
	
		
			1.5 KiB
		
	
	
	
		
			Markdown
		
	
	
	
	
	
			
		
		
	
	
			85 lines
		
	
	
		
			1.5 KiB
		
	
	
	
		
			Markdown
		
	
	
	
	
	
| ---
 | |
| description: >-
 | |
|   This guide will help install ElasticSearch connector and run manually to index
 | |
|   the metadata stored in OpenMetadata.
 | |
| ---
 | |
| 
 | |
| # ElasticSearch
 | |
| 
 | |
| {% hint style="info" %}
 | |
| **Prerequisites**
 | |
| 
 | |
| OpenMetadata is built using Java, DropWizard, Jetty, and MySQL.
 | |
| 
 | |
| 1. Python 3.7 or above
 | |
| 2. OpenMetadata Server up and running
 | |
| {% endhint %}
 | |
| 
 | |
| ### Install from PyPI or Source
 | |
| 
 | |
| {% tabs %}
 | |
| {% tab title="Install Using PyPI" %}
 | |
| ```bash
 | |
| pip install 'openmetadata-ingestion[elasticsearch]'
 | |
| ```
 | |
| {% endtab %}
 | |
| {% endtabs %}
 | |
| 
 | |
| ## Run Manually
 | |
| 
 | |
| ```bash
 | |
| metadata ingest -c ./pipelines/metadata_to_es.json
 | |
| ```
 | |
| 
 | |
| ### Configuration
 | |
| 
 | |
| {% code title="metadata_to_es.json" %}
 | |
| ```javascript
 | |
| {
 | |
|   "source": {
 | |
|     "type": "metadata_es",
 | |
|     "config": {}
 | |
|   },
 | |
| ...
 | |
| ```
 | |
| {% endcode %}
 | |
| 
 | |
| ### Publish to OpenMetadata
 | |
| 
 | |
| Below is the configuration to publish Elastic Search data into the OpenMetadata service.
 | |
| 
 | |
| Add Optionally `file` stage and `elasticsearch` bulk_sink along with `metadata-server` config
 | |
| 
 | |
| {% code title="metadata_to_es.json" %}
 | |
| ```javascript
 | |
| {
 | |
|   "source": {
 | |
|     "type": "metadata_es",
 | |
|     "config": {}
 | |
|   },
 | |
|   "stage": {
 | |
|     "type": "file",
 | |
|     "config": {
 | |
|       "filename": "/tmp/tables.txt"
 | |
|     }
 | |
|   },
 | |
|   "bulk_sink": {
 | |
|     "type": "elasticsearch",
 | |
|     "config": {
 | |
|       "filename": "/tmp/tables.txt",
 | |
|       "es_host_port": "localhost",
 | |
|       "index_name": "table_search_index"
 | |
|     }
 | |
|   },
 | |
|   "metadata_server": {
 | |
|     "type": "metadata-server",
 | |
|     "config": {
 | |
|       "api_endpoint": "http://localhost:8585/api",
 | |
|       "auth_provider_type": "no-auth"
 | |
|     }
 | |
|   }
 | |
| }
 | |
| 
 | |
| ```
 | |
| {% endcode %}
 | 
