Update ingestion connector docs

docs/install/metadata-ingestion/connectors/athena.md

@@ -5,16 +5,31 @@ description: This guide will help install Athena connector and run manually
 # Athena
 
 {% hint style="info" %}
-\*\*\*\*
 
 OpenMetadata is built using Java, DropWizard, Jetty, and MySQL.
 
 1. Python 3.7 or above
-2. Create and activate python env
-
-   ```bash
-   python3 -m venv env
-   source env/bin/activate
-   ```
 {% endhint %}
 
+
+### Install from PyPI or Source
+
+{% tabs %}
+{% tab title="Install Using PyPI" %}
+```bash
+pip install 'openmetadata-ingestion[athena]'
+```
+{% endtab %}
+{% tab title="Build from source " %}
+```bash
+# checkout OpenMetadata
+git clone https://github.com/open-metadata/OpenMetadata.git
+cd OpenMetadata/ingestion
+python3 -m venv env
+source env/bin/activate
+pip install '.[athean]'
+```
+{% endtab %}
+{% endtabs %}
+
+

docs/install/metadata-ingestion/connectors/bigquery.md

@@ -12,24 +12,32 @@ description: This guide will help install BigQuery connector and run manually
 OpenMetadata is built using Java, DropWizard, Jetty, and MySQL.
 
 1. Python 3.7 or above
-2. Create and activate python env
-
-   ```bash
-   python3 -m venv env
-   source env/bin/activate
-   ```
 {% endhint %}
 
-## Install
+### Install from PyPI or Source
 
+{% tabs %}
+{% tab title="Install Using PyPI" %}
 ```bash
-export GOOGLE_APPLICATION_CREDENTIALS="$PWD/pipelines/creds/bigquery-cred.json"
+pip install 'openmetadata-ingestion[bigquery]'
+```
+{% endtab %}
+{% tab title="Build from source " %}
+```bash
+# checkout OpenMetadata
+git clone https://github.com/open-metadata/OpenMetadata.git
+cd OpenMetadata/ingestion
+python3 -m venv env
+source env/bin/activate
 pip install '.[bigquery]'
 ```
+{% endtab %}
+{% endtabs %}
 
 ## Run Manually
 
 ```bash
+export GOOGLE_APPLICATION_CREDENTIALS="$PWD/pipelines/creds/bigquery-cred.json"
 metadata ingest -c ./pipelines/bigquery.json
 ```
 

docs/install/metadata-ingestion/connectors/elastic-search.md

@@ -12,20 +12,27 @@ description: This guide will help install ElasticSearch connector and run manual
 OpenMetadata is built using Java, DropWizard, Jetty, and MySQL.
 
 1. Python 3.7 or above
-2. Create and activate python env
-
-   ```bash
-   python3 -m venv env
-   source env/bin/activate
-   ```
 {% endhint %}
 
-## Install
+### Install from PyPI or Source
 
+{% tabs %}
+{% tab title="Install Using PyPI" %}
 ```bash
-docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.10.2
-pip install '.[elasticsearch]
+pip install 'openmetadata-ingestion[bigquery]'
 ```
+{% endtab %}
+{% tab title="Build from source " %}
+```bash
+# checkout OpenMetadata
+git clone https://github.com/open-metadata/OpenMetadata.git
+cd OpenMetadata/ingestion
+python3 -m venv env
+source env/bin/activate
+pip install '.[bigquery]'
+```
+{% endtab %}
+{% endtabs %}
 
 ## Run Manually
 

docs/install/metadata-ingestion/connectors/mssql.md

@@ -72,6 +72,10 @@ metadata ingest -c ./pipelines/mssql.json
 6. **database** - \_\*\*\_Database name from where data is to be fetched from.
 
 ## Publish to OpenMetadata
+Below is the configuration to publish mssql data into openmetadata
+
+Add Optional ```pii-tags``` processor 
+and ```metadata-rest-tables``` sink along with ```metadata-server``` config
 
 {% code title="mssql.json" %}
 ```javascript
@@ -119,5 +123,3 @@ metadata ingest -c ./pipelines/mssql.json
 ```
 {% endcode %}
 
-Add Optional ```pii-tags``` processor 
-and ```metadata-rest-tables``` sink along with ```metadata-server``` config

docs/install/metadata-ingestion/connectors/mysql.md

@@ -20,7 +20,7 @@ OpenMetadata is built using Java, DropWizard, Jetty, and MySQL.
    ```
 {% endhint %}
 
-### Install from PyPI or Source
+## Install from PyPI or Source
 
 {% tabs %}
 {% tab title="Install Using PyPI" %}
@@ -72,6 +72,10 @@ metadata ingest -c ./pipelines/mysql.json
 4. **table\_pattern** - It contains allow, deny options to choose which pattern of datasets you want to ingest into OpenMetadata
 
 ## Publish to OpenMetadata
+Below is the configuration to publish MySQL data into openmetadata
+
+Add Optional ```pii-tags``` processor 
+and ```metadata-rest-tables``` sink along with ```metadata-server``` config
 
 {% code title="mysql.json" %}
 ```javascript

docs/install/metadata-ingestion/connectors/oracle.md

@@ -5,16 +5,29 @@ description: This guide will help install Oracle connector and run manually
 # Oracle
 
 {% hint style="info" %}
-\*\*\*\*
 
 OpenMetadata is built using Java, DropWizard, Jetty, and MySQL.
 
 1. Python 3.7 or above
-2. Create and activate python env
 
-   ```bash
-   python3 -m venv env
-   source env/bin/activate
-   ```
 {% endhint %}
 
+## Install from PyPI or Source
+
+{% tabs %}
+{% tab title="Install Using PyPI" %}
+```bash
+pip install 'openmetadata-ingestion[oracle]'
+```
+{% endtab %}
+{% tab title="Build from source " %}
+```bash
+# checkout OpenMetadata
+git clone https://github.com/open-metadata/OpenMetadata.git
+cd OpenMetadata/ingestion
+python3 -m venv env
+source env/bin/activate
+pip install '.[oracle]'
+```
+{% endtab %}
+{% endtabs %}

docs/install/metadata-ingestion/connectors/postgres.md

@@ -70,3 +70,47 @@ metadata ingest -c ./pipelines/postgres.json
 4. **table\_pattern** - It contains allow, deny options to choose which pattern of datasets you want to ingest into OpenMetadata.
 5. **database -** Database name from where data is to be fetched.
 
+### Publish to OpenMetadata
+Below is the configuration to publish postgres data into openmetadata
+
+Add Optional ```pii-tags``` processor 
+and ```metadata-rest-tables``` sink along with ```metadata-server``` config
+
+{% code title="postgres.json" %}
+```javascript
+{
+  "source": {
+    "type": "postgres",
+    "config": {
+      "username": "openmetadata_user",
+      "password": "openmetadata_password",
+      "host_port": "localhost:5432",
+      "database": "pagila",
+      "service_name": "local_postgres",
+      "service_type": "Postgres"
+    }
+  },
+  "processor": {
+    "type": "pii-tags",
+    "config": {}
+  },
+  "sink": {
+    "type": "metadata-rest-tables",
+    "config": {}
+  },
+  "metadata_server": {
+    "type": "metadata-server",
+    "config": {
+      "api_endpoint": "http://localhost:8585/api",
+      "auth_provider_type": "no-auth"
+    }
+  },
+  "cron": {
+    "minute": "*/5",
+    "hour": null,
+    "day": null,
+    "month": null,
+    "day_of_week": null
+  }
+}
+```

docs/install/metadata-ingestion/connectors/redshift-usage.md

@@ -12,19 +12,27 @@ description: This guide will help install Redshift Usage connector and run manua
 OpenMetadata is built using Java, DropWizard, Jetty, and MySQL.
 
 1. Python 3.7 or above
-2. Create and activate python env
-
-   ```bash
-   python3 -m venv env
-   source env/bin/activate
-   ```
 {% endhint %}
 
-## Install
+## Install from PyPI or Source
 
+{% tabs %}
+{% tab title="Install Using PyPI" %}
 ```bash
+pip install 'openmetadata-ingestion[redshift-usage]'
+```
+{% endtab %}
+{% tab title="Build from source " %}
+```bash
+# checkout OpenMetadata
+git clone https://github.com/open-metadata/OpenMetadata.git
+cd OpenMetadata/ingestion
+python3 -m venv env
+source env/bin/activate
 pip install '.[redshift-usage]'
 ```
+{% endtab %}
+{% endtabs %}
 
 ## Run Manually
 

docs/install/metadata-ingestion/connectors/redshift.md

@@ -12,19 +12,27 @@ description: This guide will help install Redshift connector and run manually
 OpenMetadata is built using Java, DropWizard, Jetty, and MySQL.
 
 1. Python 3.7 or above
-2. Create and activate python env
-
-   ```bash
-   python3 -m venv env
-   source env/bin/activate
-   ```
 {% endhint %}
 
-## Install
+## Install from PyPI or Source
 
+{% tabs %}
+{% tab title="Install Using PyPI" %}
 ```bash
+pip install 'openmetadata-ingestion[redshift]'
+```
+{% endtab %}
+{% tab title="Build from source " %}
+```bash
+# checkout OpenMetadata
+git clone https://github.com/open-metadata/OpenMetadata.git
+cd OpenMetadata/ingestion
+python3 -m venv env
+source env/bin/activate
 pip install '.[redshift]'
 ```
+{% endtab %}
+{% endtabs %}
 
 ## Run Manually
 

docs/install/metadata-ingestion/connectors/snowflake-usage.md

@@ -12,19 +12,28 @@ description: This guide will help install Snowflake Usage connector and run manu
 OpenMetadata is built using Java, DropWizard, Jetty, and MySQL.
 
 1. Python 3.7 or above
-2. Create and activate python env
-
-   ```bash
-   python3 -m venv env
-   source env/bin/activate
-   ```
 {% endhint %}
 
-## Install
+## Install from PyPI or Source
 
+{% tabs %}
+{% tab title="Install Using PyPI" %}
 ```bash
+pip install 'openmetadata-ingestion[snowflake-usage]'
+```
+{% endtab %}
+{% tab title="Build from source " %}
+```bash
+# checkout OpenMetadata
+git clone https://github.com/open-metadata/OpenMetadata.git
+cd OpenMetadata/ingestion
+python3 -m venv env
+source env/bin/activate
 pip install '.[snowflake-usage]'
 ```
+{% endtab %}
+{% endtabs %}
+
 
 ## Run Manually
 

docs/install/metadata-ingestion/connectors/snowflake.md

@@ -12,19 +12,29 @@ description: This guide will help install Snowflake connector and run manually
 OpenMetadata is built using Java, DropWizard, Jetty, and MySQL.
 
 1. Python 3.7 or above
-2. Create and activate python env
 
-   ```bash
-   python3 -m venv env
-   source env/bin/activate
-   ```
 {% endhint %}
 
-## Install
+## Install from PyPI or Source
 
+{% tabs %}
+{% tab title="Install Using PyPI" %}
 ```bash
+pip install 'openmetadata-ingestion[snowflake]'
+```
+{% endtab %}
+{% tab title="Build from source " %}
+```bash
+# checkout OpenMetadata
+git clone https://github.com/open-metadata/OpenMetadata.git
+cd OpenMetadata/ingestion
+python3 -m venv env
+source env/bin/activate
 pip install '.[snowflake]'
 ```
+{% endtab %}
+{% endtabs %}
+
 
 ## Run Manually