yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-08-22 16:08:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

Docs Using a Custom Ingestion Image in helm (#9835)

Browse Source 
* Docs Using a Custom Ingestion Image in helm

* Update openmetadata-docs/content/deployment/kubernetes/index.md

Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>

* Update openmetadata-docs/content/deployment/kubernetes/index.md

Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>

* Update openmetadata-docs/content/deployment/kubernetes/index.md

* Update index.md

Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>
This commit is contained in:
Pere Miquel Brull 2023-01-20 10:47:55 +01:00 committed by GitHub
parent dcf220f867
commit cdd38b40d6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 54 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

54
openmetadata-docs/content/deployment/kubernetes/index.md
View File

@ -237,3 +237,57 @@ global:
```
Make sure to create RDS and OpenSearch credentials as Kubernetes Secrets mentioned [here](https://docs.open-metadata.org/deployment/kubernetes#quickstart).
## Using a Custom Ingestion Image
One possible use case where you would need to use a custom image for the ingestion is because you have developed your own custom connectors.
You can find a complete working example of this [here](https://github.com/open-metadata/openmetadata-demo/tree/main/custom-connector). After
you have your code ready, the steps would be the following:
### 1. Create a `Dockerfile` based on `openmetadata/ingestion`:
For example:
```
FROM openmetadata/ingestion:x.y.z
# Let's use the same workdir as the ingestion image
WORKDIR ingestion
USER airflow
# Install our custom connector
COPY <your_package> <your_package>
COPY setup.py .
RUN pip install --no-deps .
```
where `openmetadata/ingestion:x.y.z` needs to point to the same version of the OpenMetadata server, for example `openmetadata/ingestion:0.13.1`.
This image needs to be built and published to the container registry of your choice.
### 2. Update your dependency values YAML
The ingestion containers (which is the one shipping Airflow) gets installed in the `openmetadata-dependencies` helm chart. In this step, we use
our own custom values YAML file to point to the image we just created on the previous step. You can create a file named `values.deps.yaml` with the
following contents:
```yaml
airflow:
airflow:
image:
repository: <your repository> # by default, openmetadata/ingestion
tag: <your tag> # by default, the version you are deploying, e.g., 0.13.1
pullPolicy: "IfNotPresent"
```
### 3. Install or Upgrade openmetadata-dependencies
Upgrade/Install your openmetadata-dependencies helm charts with the below single command:
```bash
helm upgrade --install openmetadata-dependencies open-metadata/openmetadata-dependencies --values values.deps.yaml
```
### 4. Install OpenMetadata
Finally, follow the usual installation for the `openmetadata` helm charts.