OpenMetadata/docker/docker-compose-quickstart/docker-compose-postgres.yml
Sriharsha Chintalapani 5571851e53
Pluggable API/Features Limits (#16782)
* Limits

* Limits

* - Mismatched Types

* Update Limits config response

* Update Limits feature response

* Limits

* Limits

* - Mismatched Types

* Update Limits config response

* Update Limits feature response

* Limits: add entity resource enforcer

* Limits: fix rebase

* update limits enforcement

* Add OperationContext to limits

* chore: Bump versions to `1.4.0`

* chore: Bump Ingestion Versions to `1.4.0.1` for Release

* chore: Bump Ingestion Versions to `1.4.0.1`  in Dockerfiles for Release

* Remove Retry From Abstract Event Consumer (#16405)

(cherry picked from commit f8ed079731cc238dc136306fe018c5df35dd2f3b)

* Fix Migrations: Add postgres migrations (#16403)

(cherry picked from commit 9416a7ac5fa8fd9695063b108501790d813e8e6e)

* Add Null Check for isAdmin (#16407)

* Remove Retry From Abstract Event Consumer

* - Add Check for null Or Empty in isAdmin

* - Fix Test

(cherry picked from commit fe2db2d63c5495b6c288d4252a19ab77481b6de0)

* Fix OpenLineage ingestor (#16416)

* Fix OpenLineage ingestor

* py format

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>

* Minor: added whats new for 1.4.1 (#16420)

* Minor: added whats new for 1.4.1

* added note in to whats new

* Fix SSL issue (#16412)

* chore: Bump Versions for `1.4.1` Release

* chore(release): Prepare Branch for `1.4.2`

* [MINOR] partition migration issue with redshift servics (#16452)

* fix: partition migration issue with redshift servics

* chore: typo in sql comment

(cherry picked from commit 451d73593e813151c24f2c1d17efb3dcdebb71c8)

* minor(ui): update what's new for 1.4.2 (#16457)

(cherry picked from commit d55981adfd2321de706e4a043828bb473a4b05f1)

* fix:  ingestion for dbt > 1.8.0 resource_type is not an enum (#16415)

* fix: resource_type is not an enum

* feat: add log to display finis

* improve readability

* use getattr to be compatible

* format

* Add Cache Query Param for Limits

* Only Parse view  query (#16470)

* add limit check during user creation via PUT

* add limit check during user creation via PUT

* MINOR: Kafka Setup SSL Arg Fix (#16469)

* Fix#16404 - Show Node level lineage by default (#16445)

* default to node layer

* update cypress

* code cleanup

* fix cypress

(cherry picked from commit f0cda8464f34a21f45f18fa557e980fb2f105d8e)

* Invalidate count of data asset after hard delete. add limit exception to ingestion client

* - Remove Change Description from Lineage (#16488)

(cherry picked from commit 9e5c5529a84dfc781382b3a3b6abd80ee41f11f5)

* - Non Indexable fields should be remvoed at the end (#16499)

(cherry picked from commit f0b0f7a9426ca601d3bfee3989d4ce47e732a7af)

* fix announcement not redirect from landing page (#16506)

* fix announcement not redirect from landing page

* minor changes

* change in cypress test

(cherry picked from commit ee7cddd169a3a1fb1e598e80035c2fc15a5a129b)

* Fix Schema Field Null Issue (#16510)

(cherry picked from commit 022772943f1b33f6230cb35547d1da6acfaf6cfa)

* feat(ui): limits integration with application (#16206)

* feat(ui): limits integration with application

* support pipelineSchedules via limit api

* enforce limit to all the modules

* update banner styling

* update

* support disable option for ManageButton

* limit version

* fix spotlight

* update tests

* Add name and version history to resource limits
Refactor the getEntityIcon function and add new icon mappings

* limit version

* hide access token tab

* fix version for all the entity

* fix tests

* fix DQ tests

* Add fallback for the icon

* Revert the fallback icon changes

* Apply the limit to the add ingestion button in the service details page

* Fix the data quality tab add test button not working

* fix banner styling

* minor fix

* Fix ingestion component unit test

* Add InlineAlert component

* update entityNameLabels mapping object

* Fix the incorrect link in LimitBanner

* update pricing page url

* Create the GlobalSettingsClassBase

* Update URLs for pricing page and upgrade options

* fix global settings uncaught error

* add parameters to the resource limit API

* implement inline alerts for service and alert creation form

* update PRIVILEGES for docker

* fix layout issues

* fix tests

---------

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>

* Add token limitations

* Add token limitations

* Add appType as part of schema in ingestion pipeline (#16519)

* #16489: fix the redirect issue to new tab for tags and glossary (#16512)

* fix the redirect issue to new tab for tags and glossary

* fix the redirect on cancel icon and unit test issue

* changes as per comments

(cherry picked from commit 8d312f0853609cfef260739cf789d459838a3421)

* Fix  #16229 - Tag and Service filters for test cases (#16484)

* fix: added test case support for tags (inherit from table/column)]"

* feat: add tag and service filter for test cases

* feat: add tier query param

* fix: tests

(cherry picked from commit 6b00dde90285924445567ee7c396c89f0fcf3f1d)

* fix: None type is not iterable (#16496)

(cherry picked from commit 656da03b14ca24171cf7924b9dd33663e6bed423)

* minor(ui): refresh token for OIDC SSO (#16483)

* minor(ui): refresh token for OIDC SSO

* remove frame window timeout issue

* increase iFrame timeout for oidc

(cherry picked from commit 1a6c4c972052836a9b3cfa273b7ea1aa3202eafe)

* feat(ui): support tag & tier filter for test case (#16502)

* feat(ui): support tag & tier filter for test case

* fix tag filter

* allow single select for tier

* added service name filter

* update cypress for tags, tier & service

* add specific add for filters

* fix tier api call

(cherry picked from commit 5b71d79e8ac2d08a154882dfe71b9b3a0f73bffc)

* minor: sanitize activity feed editor content (#16533)

* Add appType as part of schema in ingestion pipeline (#16519)

* Fixed quicksight conn (#16537)

* fix: saml auth for new user not created (#16543)

* fix: saml auth for new user not created

* doc: add comment

* Fix#16491 -  fix lineage edge description update (#16538)

* fix lineage edge description update

* fix tests

(cherry picked from commit dff0aa8dbedcd4064ad63765cadda65bb998772e)

* CYPRESS: fix announcement cypress (#16536)

* fix announcement cypress

* changes as per comments

* fix the cypress failure

(cherry picked from commit fcb87b5866ba06aa7a6db516677e311c24053db7)

* [MINOR] Fix Test Failure for EventRegistration

* [MINOR] Fix Test Failure for EventRegistration

* [MINOR] Fix Test Failure for EventRegistration

[MINOR] Fix Test Failure for EventRegistration

* Fix Event Handlers registration Issue (#16544)

* Fix Event Handlers Issue

* Review Comments

(cherry picked from commit d374e48b7938e8ad3514dc5cf8dff619a12595e3)

* [MINOR] Fix Test Failure for EventRegistration

(cherry picked from commit 4563ad4fd10f9790c21fe744d8fc131ebd028ac8)

* Fix Topic Schema missing messageSchema (#16545)

(cherry picked from commit b612dd90c07f564d38392b1ccfe0de1505a4867b)

* Add limits exception cache in rest client

* MINOR: Ignore Cluster Information from columns (#16495)

* minor: improve the block editor initial content history (#16540)

* Minor: fixed data quality page type issue (#16556)

* #16521: fix issue in userProfilePage for roles. teams and displayName (#16527)

* fix update on roles and backlink them in user profile page

* fix teams, displayName and profile pic issue

* sonar fix

* fix cypress issue

* minor changes

(cherry picked from commit 98945cb2db87ebb325d3a72131f049abffcba345)

* Empty quick filters (#16402)

* initial commit for empty quick filters

* update progress

* fix field title

* cleanup

* add tests

* unit tests

* fix encoding of search query

* add cypress tests

* add cypress

* fix flaky cypress

* fix review comments

* revert tooltip changes

* fix tests

* fix tests

(cherry picked from commit 5930cd7a7a4bef73f6850848c85118eb64843e2d)

* Fix #16278 : Search to display Draft glossaryTerms on Explore page (#16462)

* Fix #16278 : Search to display Draft glossaryTerms as well on Explore page

* add term status quick filter

* change aggregation key for status field

* change aggregation key for status field

* add lowercase_normalizer in status filed for aggregate api

* add cypress tests

* fix cypress

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
(cherry picked from commit ae5e9d61cc9e6a39d65972987de9149a421395b1)

* [FIX] GlossaryTerm reviewers should be user or team only (#16372)

* add teams as reviewer

* Check Users to be reviewers

* Reviewers can be a team or user

* Fix check by id or name

* Review can be team or user both

* Validate Reviewers

* add multi select control

* - Fix Reviewers

* - Centralize Reviewer Relationship to EntityRepository

* - Sort

* add team as reviewer for glossary terms

* locales

* cleanup

* - Update Reviewer should remove existing reviewers

* fix selectable owner control

* fix code smells

* fix reviewer issue

* add glossary cypress

* fix patch issue on reviewers set to null

* update cypress tests

* fix cypress

* fix cypress

* fix reviewers in glossary task and supported cypress

* fix pytest

* Fix

* fix cypress

* fix code smells

* Inherited Reviewers need to be present always

* filter out inherited users

* fix cypress

* fix backend tests failure

* fix backend tests failure -checkstyle

* restrict owner to accept task in case of reviewer present

* fix pytest

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
(cherry picked from commit 9ec3d94e3b8445e63a7d77239c92c92a32536bf2)

* Add testSuite tags, domain field and check for TestCase limits

* fix owner not showing after refersh in teams page (#16567)

(cherry picked from commit 119fcf8959732a980b75e1f795a9f2dc5288cd27)

* [ISSUE-16503] Fix createUser to use EntityResource (#16549)

* Fix createUser to use EntityResource

* fix broken tests

* Fix Tests - 3

(cherry picked from commit aeb020ae3b0cbab3a2ee5995c61480cdd1eae405)

* what's new for 1.4.2 (#16568)

(cherry picked from commit c86468d9929e433922886852381269b46d69c832)

* address feedbacks

* fix error for bots page

* update banner text

* allow force fetch limit

* fix ingestion schedule

* Revert "Merge branch '1.4.2' into limits"

This reverts commit 8e965207a23ba527d0f5ba91463c1869077bf091, reversing
changes made to 4d16531965fb0d489a4afdebd45ab5b7f3d1eb5c.

* Merge 1.4.2 (#16578)

* fix explore page conflicts

* fix tests

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Chira Madlani <chirag@getcollate.io>

* fix subheader

* Updating glossary reviewers should propagate reviewers in glossary term (#16580)

* highlight inherited reviewer in glossary

* locales

* use glossary name for search query

* fix glossary version cypress

* add union datatype for subfields

* Adding reviewer to glossary also adds them as an assignee to the task

* add glossary approval cypress

---------

Co-authored-by: sonikashah <sonikashah94@gmail.com>
(cherry picked from commit 4c8bf1cac14074df87dafe7a719e2795b0a29895)

* Update documentation for Search Index apis (#16539)

(cherry picked from commit d3123c49143652015c416d271d9fd0f9cfa9e324)

* cypress: fixed flakiness and announcment cypress (#16579)

* fetch latest limit for create / delete operations

* guard datAsset limit got topic, dashboard, mlmodel etc

* Fix: Ensure correct index mapping in Elasticsearch for clusterAlias (#16589)

* Fix: Ensure correct index mapping in Elasticsearch for clusterAlias

* Fix: Ensure correct index mapping in Elasticsearch for clusterAlias

(cherry picked from commit 8723b8c36afe31410c31d1ebbdafe7b1770921fa)

* cypress: fixed cypress AUT for mysql (#16446)

* cypress: fixed cypress AUT for mysql

* minor fix

* skip announcment redirection cypress

* Minor: Ensure correct index mapping in Elasticsearch for clusterAlias (#16598)

(cherry picked from commit 04543722a6f6e2b1eaf7a451ebb1c176862bc346)

* Fix Postgres Application listing (#16600)

* Fix Postgres Application listing

* Fix Listing

(cherry picked from commit 77dfe1f6af53d187ff7a61fdb1e1416de7178f5a)

* fix limit related issue

* Fix Automations limits invalidation during the uninstall

* cypress: fixed 1.4.2 AUT cypress (#16602)

* cypress: fixed 1.4.2 AUT cypress

* fix cypress around announcement,user,glossary, lineage and mydata

* searchIndexApplication fix and minor changes

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>

* test: add updateJWTTokenExpiryTime util (#16606)

(cherry picked from commit 8c173bed6a279cb0a648bd30632ea6ebdf4a2a90)

* OSS changes for adding automator cypress tests (#16611)

* Fix Test Suite Filter (#16615)

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
(cherry picked from commit 3db41f08e27f388495040e5b23cc7bee5ae665f1)

* MINOR: Fix Profiler for SSL Enabled Source (#16613)

* Add Test Suite SSL (#16619)

* MINOR: Fix ssl connection in usage & lineage (#16625)

* Fix owner notification (#16629)

* - Fix Task notification not getting sent to owners

* - Fix Task notification not getting sent to owners

(cherry picked from commit cc2d581eb0524604b6dcf0523e9ca96e0b8a6ce3)

* chore(release): Prepare Branch for `1.4.3`

* - Fix User Signup (#16667)

(cherry picked from commit b4cba8a850ecd7a25aeff6ca7dea0dc432d43d86)

* - Fix User Signup - p2

(cherry picked from commit d9ae6f6db9891f8e9bf7ad49c561a71dd50103da)

* - Update What's new (#16669)

- fix vulnerability

(cherry picked from commit 1dcb1bd46f9da49764f4c61a7ac5048dd2fa956b)

* Minor: Fix incorrect alert on signup page (#16666)

* Fix Application enforceLimits during install

* Wrap the add test button with limits wrapper for column profile tab

* fix errors

* fix tests

* fix pylint

* fix tests

* fix limits

* pylint

* fix schedule options

* fix glossary spec failure

* Add domain & tags to testSuite

* Update airflow-apis-tests-3_9.yml

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Chira Madlani <chirag@getcollate.io>
Co-authored-by: Pablo Takara <pjt1991@gmail.com>
Co-authored-by: Akash-Jain <15995028+akash-jain-10@users.noreply.github.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Maxim Martynov <martinov_m_s_@mail.ru>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Antoine Balliet <antoine.balliet@gorgias.com>
Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2024-07-01 14:59:25 +05:30

534 lines
28 KiB
YAML

# Copyright 2021 Collate
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
version: "3.9"
volumes:
ingestion-volume-dag-airflow:
ingestion-volume-dags:
ingestion-volume-tmp:
es-data:
services:
postgresql:
container_name: openmetadata_postgresql
image: docker.getcollate.io/openmetadata/postgresql:1.4.3
restart: always
command: "--work_mem=10MB"
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: password
expose:
- 5432
ports:
- "5432:5432"
volumes:
- ./docker-volume/db-data-postgres:/var/lib/postgresql/data
networks:
- app_net
healthcheck:
test: psql -U postgres -tAc 'select 1' -d openmetadata_db
interval: 15s
timeout: 10s
retries: 10
elasticsearch:
container_name: openmetadata_elasticsearch
image: docker.elastic.co/elasticsearch/elasticsearch:8.10.2
environment:
- discovery.type=single-node
- ES_JAVA_OPTS=-Xms1024m -Xmx1024m
- xpack.security.enabled=false
networks:
- app_net
ports:
- "9200:9200"
- "9300:9300"
healthcheck:
test: "curl -s http://localhost:9200/_cluster/health?pretty | grep status | grep -qE 'green|yellow' || exit 1"
interval: 15s
timeout: 10s
retries: 10
volumes:
- es-data:/usr/share/elasticsearch/data
execute-migrate-all:
container_name: execute_migrate_all
image: docker.getcollate.io/openmetadata/server:1.4.3
command: "./bootstrap/openmetadata-ops.sh migrate"
environment:
OPENMETADATA_CLUSTER_NAME: ${OPENMETADATA_CLUSTER_NAME:-openmetadata}
SERVER_PORT: ${SERVER_PORT:-8585}
SERVER_ADMIN_PORT: ${SERVER_ADMIN_PORT:-8586}
LOG_LEVEL: ${LOG_LEVEL:-INFO}
# Migration
MIGRATION_LIMIT_PARAM: ${MIGRATION_LIMIT_PARAM:-1200}
# OpenMetadata Server Authentication Configuration
AUTHORIZER_CLASS_NAME: ${AUTHORIZER_CLASS_NAME:-org.openmetadata.service.security.DefaultAuthorizer}
AUTHORIZER_REQUEST_FILTER: ${AUTHORIZER_REQUEST_FILTER:-org.openmetadata.service.security.JwtFilter}
AUTHORIZER_ADMIN_PRINCIPALS: ${AUTHORIZER_ADMIN_PRINCIPALS:-[admin]}
AUTHORIZER_ALLOWED_REGISTRATION_DOMAIN: ${AUTHORIZER_ALLOWED_REGISTRATION_DOMAIN:-["all"]}
AUTHORIZER_INGESTION_PRINCIPALS: ${AUTHORIZER_INGESTION_PRINCIPALS:-[ingestion-bot]}
AUTHORIZER_PRINCIPAL_DOMAIN: ${AUTHORIZER_PRINCIPAL_DOMAIN:-"openmetadata.org"}
AUTHORIZER_ENFORCE_PRINCIPAL_DOMAIN: ${AUTHORIZER_ENFORCE_PRINCIPAL_DOMAIN:-false}
AUTHORIZER_ENABLE_SECURE_SOCKET: ${AUTHORIZER_ENABLE_SECURE_SOCKET:-false}
AUTHENTICATION_PROVIDER: ${AUTHENTICATION_PROVIDER:-basic}
AUTHENTICATION_RESPONSE_TYPE: ${AUTHENTICATION_RESPONSE_TYPE:-id_token}
CUSTOM_OIDC_AUTHENTICATION_PROVIDER_NAME: ${CUSTOM_OIDC_AUTHENTICATION_PROVIDER_NAME:-""}
AUTHENTICATION_PUBLIC_KEYS: ${AUTHENTICATION_PUBLIC_KEYS:-[http://localhost:8585/api/v1/system/config/jwks]}
AUTHENTICATION_AUTHORITY: ${AUTHENTICATION_AUTHORITY:-https://accounts.google.com}
AUTHENTICATION_CLIENT_ID: ${AUTHENTICATION_CLIENT_ID:-""}
AUTHENTICATION_CALLBACK_URL: ${AUTHENTICATION_CALLBACK_URL:-""}
AUTHENTICATION_JWT_PRINCIPAL_CLAIMS: ${AUTHENTICATION_JWT_PRINCIPAL_CLAIMS:-[email,preferred_username,sub]}
AUTHENTICATION_JWT_PRINCIPAL_CLAIMS_MAPPING: ${AUTHENTICATION_JWT_PRINCIPAL_CLAIMS_MAPPING:-[]}
AUTHENTICATION_ENABLE_SELF_SIGNUP: ${AUTHENTICATION_ENABLE_SELF_SIGNUP:-true}
AUTHENTICATION_CLIENT_TYPE: ${AUTHENTICATION_CLIENT_TYPE:-public}
#For OIDC Authentication, when client is confidential
OIDC_CLIENT_ID: ${OIDC_CLIENT_ID:-""}
OIDC_TYPE: ${OIDC_TYPE:-""} # google, azure etc.
OIDC_CLIENT_SECRET: ${OIDC_CLIENT_SECRET:-""}
OIDC_SCOPE: ${OIDC_SCOPE:-"openid email profile"}
OIDC_DISCOVERY_URI: ${OIDC_DISCOVERY_URI:-""}
OIDC_USE_NONCE: ${OIDC_USE_NONCE:-true}
OIDC_PREFERRED_JWS: ${OIDC_PREFERRED_JWS:-"RS256"}
OIDC_RESPONSE_TYPE: ${OIDC_RESPONSE_TYPE:-"code"}
OIDC_DISABLE_PKCE: ${OIDC_DISABLE_PKCE:-true}
OIDC_CALLBACK: ${OIDC_CALLBACK:-"http://localhost:8585/callback"}
OIDC_SERVER_URL: ${OIDC_SERVER_URL:-"http://localhost:8585"}
OIDC_CLIENT_AUTH_METHOD: ${OIDC_CLIENT_AUTH_METHOD:-"client_secret_post"}
OIDC_TENANT: ${OIDC_TENANT:-""}
OIDC_MAX_CLOCK_SKEW: ${OIDC_MAX_CLOCK_SKEW:-""}
OIDC_CUSTOM_PARAMS: ${OIDC_CUSTOM_PARAMS:-{}}
# For SAML Authentication
# SAML_DEBUG_MODE: ${SAML_DEBUG_MODE:-false}
# SAML_IDP_ENTITY_ID: ${SAML_IDP_ENTITY_ID:-""}
# SAML_IDP_SSO_LOGIN_URL: ${SAML_IDP_SSO_LOGIN_URL:-""}
# SAML_IDP_CERTIFICATE: ${SAML_IDP_CERTIFICATE:-""}
# SAML_AUTHORITY_URL: ${SAML_AUTHORITY_URL:-"http://localhost:8585/api/v1/saml/login"}
# SAML_IDP_NAME_ID: ${SAML_IDP_NAME_ID:-"urn:oasis:names:tc:SAML:2.0:nameid-format:emailAddress"}
# SAML_SP_ENTITY_ID: ${SAML_SP_ENTITY_ID:-"http://localhost:8585/api/v1/saml/metadata"}
# SAML_SP_ACS: ${SAML_SP_ACS:-"http://localhost:8585/api/v1/saml/acs"}
# SAML_SP_CERTIFICATE: ${SAML_SP_CERTIFICATE:-""}
# SAML_SP_CALLBACK: ${SAML_SP_CALLBACK:-"http://localhost:8585/saml/callback"}
# SAML_STRICT_MODE: ${SAML_STRICT_MODE:-false}
# SAML_SP_TOKEN_VALIDITY: ${SAML_SP_TOKEN_VALIDITY:-"3600"}
# SAML_SEND_ENCRYPTED_NAME_ID: ${SAML_SEND_ENCRYPTED_NAME_ID:-false}
# SAML_SEND_SIGNED_AUTH_REQUEST: ${SAML_SEND_SIGNED_AUTH_REQUEST:-false}
# SAML_SIGNED_SP_METADATA: ${SAML_SIGNED_SP_METADATA:-false}
# SAML_WANT_MESSAGE_SIGNED: ${SAML_WANT_MESSAGE_SIGNED:-false}
# SAML_WANT_ASSERTION_SIGNED: ${SAML_WANT_ASSERTION_SIGNED:-false}
# SAML_WANT_ASSERTION_ENCRYPTED: ${SAML_WANT_ASSERTION_ENCRYPTED:-false}
# SAML_WANT_NAME_ID_ENCRYPTED: ${SAML_WANT_NAME_ID_ENCRYPTED:-false}
# SAML_KEYSTORE_FILE_PATH: ${SAML_KEYSTORE_FILE_PATH:-""}
# SAML_KEYSTORE_ALIAS: ${SAML_KEYSTORE_ALIAS:-""}
# SAML_KEYSTORE_PASSWORD: ${SAML_KEYSTORE_PASSWORD:-""}
# For LDAP Authentication
# AUTHENTICATION_LDAP_HOST: ${AUTHENTICATION_LDAP_HOST:-}
# AUTHENTICATION_LDAP_PORT: ${AUTHENTICATION_LDAP_PORT:-}
# AUTHENTICATION_LOOKUP_ADMIN_DN: ${AUTHENTICATION_LOOKUP_ADMIN_DN:-""}
# AUTHENTICATION_LOOKUP_ADMIN_PWD: ${AUTHENTICATION_LOOKUP_ADMIN_PWD:-""}
# AUTHENTICATION_USER_LOOKUP_BASEDN: ${AUTHENTICATION_USER_LOOKUP_BASEDN:-""}
# AUTHENTICATION_USER_MAIL_ATTR: ${AUTHENTICATION_USER_MAIL_ATTR:-}
# AUTHENTICATION_LDAP_POOL_SIZE: ${AUTHENTICATION_LDAP_POOL_SIZE:-3}
# AUTHENTICATION_LDAP_SSL_ENABLED: ${AUTHENTICATION_LDAP_SSL_ENABLED:-}
# AUTHENTICATION_LDAP_TRUSTSTORE_TYPE: ${AUTHENTICATION_LDAP_TRUSTSTORE_TYPE:-TrustAll}
# AUTHENTICATION_LDAP_TRUSTSTORE_PATH: ${AUTHENTICATION_LDAP_TRUSTSTORE_PATH:-}
# AUTHENTICATION_LDAP_KEYSTORE_PASSWORD: ${AUTHENTICATION_LDAP_KEYSTORE_PASSWORD:-}
# AUTHENTICATION_LDAP_SSL_KEY_FORMAT: ${AUTHENTICATION_LDAP_SSL_KEY_FORMAT:-}
# AUTHENTICATION_LDAP_ALLOW_WILDCARDS: ${AUTHENTICATION_LDAP_ALLOW_WILDCARDS:-}
# AUTHENTICATION_LDAP_ALLOWED_HOSTNAMES: ${AUTHENTICATION_LDAP_ALLOWED_HOSTNAMES:-[]}
# AUTHENTICATION_LDAP_SSL_VERIFY_CERT_HOST: ${AUTHENTICATION_LDAP_SSL_VERIFY_CERT_HOST:-}
# AUTHENTICATION_LDAP_EXAMINE_VALIDITY_DATES: ${AUTHENTICATION_LDAP_EXAMINE_VALIDITY_DATES:-true}
# JWT Configuration
RSA_PUBLIC_KEY_FILE_PATH: ${RSA_PUBLIC_KEY_FILE_PATH:-"./conf/public_key.der"}
RSA_PRIVATE_KEY_FILE_PATH: ${RSA_PRIVATE_KEY_FILE_PATH:-"./conf/private_key.der"}
JWT_ISSUER: ${JWT_ISSUER:-"open-metadata.org"}
JWT_KEY_ID: ${JWT_KEY_ID:-"Gb389a-9f76-gdjs-a92j-0242bk94356"}
# OpenMetadata Server Pipeline Service Client Configuration
PIPELINE_SERVICE_CLIENT_ENDPOINT: ${PIPELINE_SERVICE_CLIENT_ENDPOINT:-http://ingestion:8080}
PIPELINE_SERVICE_CLIENT_HEALTH_CHECK_INTERVAL: ${PIPELINE_SERVICE_CLIENT_HEALTH_CHECK_INTERVAL:-300}
SERVER_HOST_API_URL: ${SERVER_HOST_API_URL:-http://openmetadata-server:8585/api}
PIPELINE_SERVICE_CLIENT_VERIFY_SSL: ${PIPELINE_SERVICE_CLIENT_VERIFY_SSL:-"no-ssl"}
PIPELINE_SERVICE_CLIENT_SSL_CERT_PATH: ${PIPELINE_SERVICE_CLIENT_SSL_CERT_PATH:-""}
#Database configuration for postgresql
DB_DRIVER_CLASS: ${DB_DRIVER_CLASS:-org.postgresql.Driver}
DB_SCHEME: ${DB_SCHEME:-postgresql}
DB_PARAMS: ${DB_PARAMS:-allowPublicKeyRetrieval=true&useSSL=false&serverTimezone=UTC}
DB_USER: ${DB_USER:-openmetadata_user}
DB_USER_PASSWORD: ${DB_USER_PASSWORD:-openmetadata_password}
DB_HOST: ${DB_HOST:-postgresql}
DB_PORT: ${DB_PORT:-5432}
OM_DATABASE: ${OM_DATABASE:-openmetadata_db}
# ElasticSearch Configurations
ELASTICSEARCH_HOST: ${ELASTICSEARCH_HOST:- elasticsearch}
ELASTICSEARCH_PORT: ${ELASTICSEARCH_PORT:-9200}
ELASTICSEARCH_SCHEME: ${ELASTICSEARCH_SCHEME:-http}
ELASTICSEARCH_USER: ${ELASTICSEARCH_USER:-""}
ELASTICSEARCH_PASSWORD: ${ELASTICSEARCH_PASSWORD:-""}
SEARCH_TYPE: ${SEARCH_TYPE:- "elasticsearch"}
ELASTICSEARCH_TRUST_STORE_PATH: ${ELASTICSEARCH_TRUST_STORE_PATH:-""}
ELASTICSEARCH_TRUST_STORE_PASSWORD: ${ELASTICSEARCH_TRUST_STORE_PASSWORD:-""}
ELASTICSEARCH_CONNECTION_TIMEOUT_SECS: ${ELASTICSEARCH_CONNECTION_TIMEOUT_SECS:-5}
ELASTICSEARCH_SOCKET_TIMEOUT_SECS: ${ELASTICSEARCH_SOCKET_TIMEOUT_SECS:-60}
ELASTICSEARCH_KEEP_ALIVE_TIMEOUT_SECS: ${ELASTICSEARCH_KEEP_ALIVE_TIMEOUT_SECS:-600}
ELASTICSEARCH_BATCH_SIZE: ${ELASTICSEARCH_BATCH_SIZE:-10}
ELASTICSEARCH_PAYLOAD_BYTES_SIZE: ${ELASTICSEARCH_PAYLOAD_BYTES_SIZE:-10485760} #max payLoadSize in Bytes
ELASTICSEARCH_INDEX_MAPPING_LANG: ${ELASTICSEARCH_INDEX_MAPPING_LANG:-EN}
#eventMonitoringConfiguration
EVENT_MONITOR: ${EVENT_MONITOR:-prometheus}
EVENT_MONITOR_BATCH_SIZE: ${EVENT_MONITOR_BATCH_SIZE:-10}
EVENT_MONITOR_PATH_PATTERN: ${EVENT_MONITOR_PATH_PATTERN:-["/api/v1/tables/*", "/api/v1/health-check"]}
EVENT_MONITOR_LATENCY: ${EVENT_MONITOR_LATENCY:-[]}
#pipelineServiceClientConfiguration
PIPELINE_SERVICE_CLIENT_ENABLED: ${PIPELINE_SERVICE_CLIENT_ENABLED:-true}
PIPELINE_SERVICE_CLIENT_CLASS_NAME: ${PIPELINE_SERVICE_CLIENT_CLASS_NAME:-"org.openmetadata.service.clients.pipeline.airflow.AirflowRESTClient"}
PIPELINE_SERVICE_IP_INFO_ENABLED: ${PIPELINE_SERVICE_IP_INFO_ENABLED:-false}
PIPELINE_SERVICE_CLIENT_HOST_IP: ${PIPELINE_SERVICE_CLIENT_HOST_IP:-""}
PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER: ${PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER:-"noop"}
#airflow parameters
AIRFLOW_USERNAME: ${AIRFLOW_USERNAME:-admin}
AIRFLOW_PASSWORD: ${AIRFLOW_PASSWORD:-admin}
AIRFLOW_TIMEOUT: ${AIRFLOW_TIMEOUT:-10}
AIRFLOW_TRUST_STORE_PATH: ${AIRFLOW_TRUST_STORE_PATH:-""}
AIRFLOW_TRUST_STORE_PASSWORD: ${AIRFLOW_TRUST_STORE_PASSWORD:-""}
FERNET_KEY: ${FERNET_KEY:-jJ/9sz0g0OHxsfxOoSfdFdmk3ysNmPRnH3TUAbz3IHA=}
#secretsManagerConfiguration
SECRET_MANAGER: ${SECRET_MANAGER:-db}
# AWS:
OM_SM_REGION: ${OM_SM_REGION:-""}
OM_SM_ACCESS_KEY_ID: ${OM_SM_ACCESS_KEY_ID:-""}
OM_SM_ACCESS_KEY: ${OM_SM_ACCESS_KEY:-""}
# Azure:
OM_SM_VAULT_NAME: ${OM_SM_VAULT_NAME:-""}
OM_SM_CLIENT_ID: ${OM_SM_CLIENT_ID:-""}
OM_SM_CLIENT_SECRET: ${OM_SM_CLIENT_SECRET:-""}
OM_SM_TENANT_ID: ${OM_SM_TENANT_ID:-""}
#email configuration:
OM_EMAIL_ENTITY: ${OM_EMAIL_ENTITY:-"OpenMetadata"}
OM_SUPPORT_URL: ${OM_SUPPORT_URL:-"https://slack.open-metadata.org"}
AUTHORIZER_ENABLE_SMTP : ${AUTHORIZER_ENABLE_SMTP:-false}
OPENMETADATA_SERVER_URL: ${OPENMETADATA_SERVER_URL:-""}
OPENMETADATA_SMTP_SENDER_MAIL: ${OPENMETADATA_SMTP_SENDER_MAIL:-""}
SMTP_SERVER_ENDPOINT: ${SMTP_SERVER_ENDPOINT:-""}
SMTP_SERVER_PORT: ${SMTP_SERVER_PORT:-""}
SMTP_SERVER_USERNAME: ${SMTP_SERVER_USERNAME:-""}
SMTP_SERVER_PWD: ${SMTP_SERVER_PWD:-""}
SMTP_SERVER_STRATEGY: ${SMTP_SERVER_STRATEGY:-"SMTP_TLS"}
# Heap OPTS Configurations
OPENMETADATA_HEAP_OPTS: ${OPENMETADATA_HEAP_OPTS:--Xmx1G -Xms1G}
# Mask passwords values in UI
MASK_PASSWORDS_API: ${MASK_PASSWORDS_API:-false}
#OpenMetadata Web Configuration
WEB_CONF_URI_PATH: ${WEB_CONF_URI_PATH:-"/api"}
#HSTS
WEB_CONF_HSTS_ENABLED: ${WEB_CONF_HSTS_ENABLED:-false}
WEB_CONF_HSTS_MAX_AGE: ${WEB_CONF_HSTS_MAX_AGE:-"365 days"}
WEB_CONF_HSTS_INCLUDE_SUBDOMAINS: ${WEB_CONF_HSTS_INCLUDE_SUBDOMAINS:-"true"}
WEB_CONF_HSTS_PRELOAD: ${WEB_CONF_HSTS_PRELOAD:-"true"}
#Frame Options
WEB_CONF_FRAME_OPTION_ENABLED: ${WEB_CONF_FRAME_OPTION_ENABLED:-false}
WEB_CONF_FRAME_OPTION: ${WEB_CONF_FRAME_OPTION:-"SAMEORIGIN"}
WEB_CONF_FRAME_ORIGIN: ${WEB_CONF_FRAME_ORIGIN:-""}
#Content Type
WEB_CONF_CONTENT_TYPE_OPTIONS_ENABLED: ${WEB_CONF_CONTENT_TYPE_OPTIONS_ENABLED:-false}
#XSS-Protection
WEB_CONF_XSS_PROTECTION_ENABLED: ${WEB_CONF_XSS_PROTECTION_ENABLED:-false}
WEB_CONF_XSS_PROTECTION_ON: ${WEB_CONF_XSS_PROTECTION_ON:-true}
WEB_CONF_XSS_PROTECTION_BLOCK: ${WEB_CONF_XSS_PROTECTION_BLOCK:-true}
#CSP
WEB_CONF_XSS_CSP_ENABLED: ${WEB_CONF_XSS_CSP_ENABLED:-false}
WEB_CONF_XSS_CSP_POLICY: ${WEB_CONF_XSS_CSP_POLICY:-"default-src 'self'"}
WEB_CONF_XSS_CSP_REPORT_ONLY_POLICY: ${WEB_CONF_XSS_CSP_REPORT_ONLY_POLICY:-""}
#Referrer-Policy
WEB_CONF_REFERRER_POLICY_ENABLED: ${WEB_CONF_REFERRER_POLICY_ENABLED:-false}
WEB_CONF_REFERRER_POLICY_OPTION: ${WEB_CONF_REFERRER_POLICY_OPTION:-"SAME_ORIGIN"}
#Permission-Policy
WEB_CONF_PERMISSION_POLICY_ENABLED: ${WEB_CONF_PERMISSION_POLICY_ENABLED:-false}
WEB_CONF_PERMISSION_POLICY_OPTION: ${WEB_CONF_PERMISSION_POLICY_OPTION:-""}
#Cache
WEB_CONF_CACHE_CONTROL: ${WEB_CONF_CACHE_CONTROL:-""}
WEB_CONF_PRAGMA: ${WEB_CONF_PRAGMA:-""}
depends_on:
elasticsearch:
condition: service_healthy
postgresql:
condition: service_healthy
networks:
- app_net
openmetadata-server:
container_name: openmetadata_server
restart: always
image: docker.getcollate.io/openmetadata/server:1.4.3
environment:
OPENMETADATA_CLUSTER_NAME: ${OPENMETADATA_CLUSTER_NAME:-openmetadata}
SERVER_PORT: ${SERVER_PORT:-8585}
SERVER_ADMIN_PORT: ${SERVER_ADMIN_PORT:-8586}
LOG_LEVEL: ${LOG_LEVEL:-INFO}
# OpenMetadata Server Authentication Configuration
AUTHORIZER_CLASS_NAME: ${AUTHORIZER_CLASS_NAME:-org.openmetadata.service.security.DefaultAuthorizer}
AUTHORIZER_REQUEST_FILTER: ${AUTHORIZER_REQUEST_FILTER:-org.openmetadata.service.security.JwtFilter}
AUTHORIZER_ADMIN_PRINCIPALS: ${AUTHORIZER_ADMIN_PRINCIPALS:-[admin]}
AUTHORIZER_ALLOWED_REGISTRATION_DOMAIN: ${AUTHORIZER_ALLOWED_REGISTRATION_DOMAIN:-["all"]}
AUTHORIZER_INGESTION_PRINCIPALS: ${AUTHORIZER_INGESTION_PRINCIPALS:-[ingestion-bot]}
AUTHORIZER_PRINCIPAL_DOMAIN: ${AUTHORIZER_PRINCIPAL_DOMAIN:-"openmetadata.org"}
AUTHORIZER_ENFORCE_PRINCIPAL_DOMAIN: ${AUTHORIZER_ENFORCE_PRINCIPAL_DOMAIN:-false}
AUTHORIZER_ENABLE_SECURE_SOCKET: ${AUTHORIZER_ENABLE_SECURE_SOCKET:-false}
AUTHENTICATION_PROVIDER: ${AUTHENTICATION_PROVIDER:-basic}
AUTHENTICATION_RESPONSE_TYPE: ${AUTHENTICATION_RESPONSE_TYPE:-id_token}
CUSTOM_OIDC_AUTHENTICATION_PROVIDER_NAME: ${CUSTOM_OIDC_AUTHENTICATION_PROVIDER_NAME:-""}
AUTHENTICATION_PUBLIC_KEYS: ${AUTHENTICATION_PUBLIC_KEYS:-[http://localhost:8585/api/v1/system/config/jwks]}
AUTHENTICATION_AUTHORITY: ${AUTHENTICATION_AUTHORITY:-https://accounts.google.com}
AUTHENTICATION_CLIENT_ID: ${AUTHENTICATION_CLIENT_ID:-""}
AUTHENTICATION_CALLBACK_URL: ${AUTHENTICATION_CALLBACK_URL:-""}
AUTHENTICATION_JWT_PRINCIPAL_CLAIMS: ${AUTHENTICATION_JWT_PRINCIPAL_CLAIMS:-[email,preferred_username,sub]}
AUTHENTICATION_JWT_PRINCIPAL_CLAIMS_MAPPING: ${AUTHENTICATION_JWT_PRINCIPAL_CLAIMS_MAPPING:-[]}
AUTHENTICATION_ENABLE_SELF_SIGNUP: ${AUTHENTICATION_ENABLE_SELF_SIGNUP:-true}
AUTHENTICATION_CLIENT_TYPE: ${AUTHENTICATION_CLIENT_TYPE:-public}
#For OIDC Authentication, when client is confidential
OIDC_CLIENT_ID: ${OIDC_CLIENT_ID:-""}
OIDC_TYPE: ${OIDC_TYPE:-""} # google, azure etc.
OIDC_CLIENT_SECRET: ${OIDC_CLIENT_SECRET:-""}
OIDC_SCOPE: ${OIDC_SCOPE:-"openid email profile"}
OIDC_DISCOVERY_URI: ${OIDC_DISCOVERY_URI:-""}
OIDC_USE_NONCE: ${OIDC_USE_NONCE:-true}
OIDC_PREFERRED_JWS: ${OIDC_PREFERRED_JWS:-"RS256"}
OIDC_RESPONSE_TYPE: ${OIDC_RESPONSE_TYPE:-"code"}
OIDC_DISABLE_PKCE: ${OIDC_DISABLE_PKCE:-true}
OIDC_CALLBACK: ${OIDC_CALLBACK:-"http://localhost:8585/callback"}
OIDC_SERVER_URL: ${OIDC_SERVER_URL:-"http://localhost:8585"}
OIDC_CLIENT_AUTH_METHOD: ${OIDC_CLIENT_AUTH_METHOD:-"client_secret_post"}
OIDC_TENANT: ${OIDC_TENANT:-""}
OIDC_MAX_CLOCK_SKEW: ${OIDC_MAX_CLOCK_SKEW:-""}
OIDC_CUSTOM_PARAMS: ${OIDC_CUSTOM_PARAMS:-{}}
# For SAML Authentication
# SAML_DEBUG_MODE: ${SAML_DEBUG_MODE:-false}
# SAML_IDP_ENTITY_ID: ${SAML_IDP_ENTITY_ID:-""}
# SAML_IDP_SSO_LOGIN_URL: ${SAML_IDP_SSO_LOGIN_URL:-""}
# SAML_IDP_CERTIFICATE: ${SAML_IDP_CERTIFICATE:-""}
# SAML_AUTHORITY_URL: ${SAML_AUTHORITY_URL:-"http://localhost:8585/api/v1/saml/login"}
# SAML_IDP_NAME_ID: ${SAML_IDP_NAME_ID:-"urn:oasis:names:tc:SAML:2.0:nameid-format:emailAddress"}
# SAML_SP_ENTITY_ID: ${SAML_SP_ENTITY_ID:-"http://localhost:8585/api/v1/saml/metadata"}
# SAML_SP_ACS: ${SAML_SP_ACS:-"http://localhost:8585/api/v1/saml/acs"}
# SAML_SP_CERTIFICATE: ${SAML_SP_CERTIFICATE:-""}
# SAML_SP_CALLBACK: ${SAML_SP_CALLBACK:-"http://localhost:8585/saml/callback"}
# SAML_STRICT_MODE: ${SAML_STRICT_MODE:-false}
# SAML_SP_TOKEN_VALIDITY: ${SAML_SP_TOKEN_VALIDITY:-"3600"}
# SAML_SEND_ENCRYPTED_NAME_ID: ${SAML_SEND_ENCRYPTED_NAME_ID:-false}
# SAML_SEND_SIGNED_AUTH_REQUEST: ${SAML_SEND_SIGNED_AUTH_REQUEST:-false}
# SAML_SIGNED_SP_METADATA: ${SAML_SIGNED_SP_METADATA:-false}
# SAML_WANT_MESSAGE_SIGNED: ${SAML_WANT_MESSAGE_SIGNED:-false}
# SAML_WANT_ASSERTION_SIGNED: ${SAML_WANT_ASSERTION_SIGNED:-false}
# SAML_WANT_ASSERTION_ENCRYPTED: ${SAML_WANT_ASSERTION_ENCRYPTED:-false}
# SAML_WANT_NAME_ID_ENCRYPTED: ${SAML_WANT_NAME_ID_ENCRYPTED:-false}
# SAML_KEYSTORE_FILE_PATH: ${SAML_KEYSTORE_FILE_PATH:-""}
# SAML_KEYSTORE_ALIAS: ${SAML_KEYSTORE_ALIAS:-""}
# SAML_KEYSTORE_PASSWORD: ${SAML_KEYSTORE_PASSWORD:-""}
# For LDAP Authentication
# AUTHENTICATION_LDAP_HOST: ${AUTHENTICATION_LDAP_HOST:-}
# AUTHENTICATION_LDAP_PORT: ${AUTHENTICATION_LDAP_PORT:-}
# AUTHENTICATION_LOOKUP_ADMIN_DN: ${AUTHENTICATION_LOOKUP_ADMIN_DN:-""}
# AUTHENTICATION_LOOKUP_ADMIN_PWD: ${AUTHENTICATION_LOOKUP_ADMIN_PWD:-""}
# AUTHENTICATION_USER_LOOKUP_BASEDN: ${AUTHENTICATION_USER_LOOKUP_BASEDN:-""}
# AUTHENTICATION_USER_MAIL_ATTR: ${AUTHENTICATION_USER_MAIL_ATTR:-}
# AUTHENTICATION_LDAP_POOL_SIZE: ${AUTHENTICATION_LDAP_POOL_SIZE:-3}
# AUTHENTICATION_LDAP_SSL_ENABLED: ${AUTHENTICATION_LDAP_SSL_ENABLED:-}
# AUTHENTICATION_LDAP_TRUSTSTORE_TYPE: ${AUTHENTICATION_LDAP_TRUSTSTORE_TYPE:-TrustAll}
# AUTHENTICATION_LDAP_TRUSTSTORE_PATH: ${AUTHENTICATION_LDAP_TRUSTSTORE_PATH:-}
# AUTHENTICATION_LDAP_KEYSTORE_PASSWORD: ${AUTHENTICATION_LDAP_KEYSTORE_PASSWORD:-}
# AUTHENTICATION_LDAP_SSL_KEY_FORMAT: ${AUTHENTICATION_LDAP_SSL_KEY_FORMAT:-}
# AUTHENTICATION_LDAP_ALLOW_WILDCARDS: ${AUTHENTICATION_LDAP_ALLOW_WILDCARDS:-}
# AUTHENTICATION_LDAP_ALLOWED_HOSTNAMES: ${AUTHENTICATION_LDAP_ALLOWED_HOSTNAMES:-[]}
# AUTHENTICATION_LDAP_SSL_VERIFY_CERT_HOST: ${AUTHENTICATION_LDAP_SSL_VERIFY_CERT_HOST:-}
# AUTHENTICATION_LDAP_EXAMINE_VALIDITY_DATES: ${AUTHENTICATION_LDAP_EXAMINE_VALIDITY_DATES:-true}
# JWT Configuration
RSA_PUBLIC_KEY_FILE_PATH: ${RSA_PUBLIC_KEY_FILE_PATH:-"./conf/public_key.der"}
RSA_PRIVATE_KEY_FILE_PATH: ${RSA_PRIVATE_KEY_FILE_PATH:-"./conf/private_key.der"}
JWT_ISSUER: ${JWT_ISSUER:-"open-metadata.org"}
JWT_KEY_ID: ${JWT_KEY_ID:-"Gb389a-9f76-gdjs-a92j-0242bk94356"}
# OpenMetadata Server Pipeline Service Client Configuration
PIPELINE_SERVICE_CLIENT_ENDPOINT: ${PIPELINE_SERVICE_CLIENT_ENDPOINT:-http://ingestion:8080}
PIPELINE_SERVICE_CLIENT_HEALTH_CHECK_INTERVAL: ${PIPELINE_SERVICE_CLIENT_HEALTH_CHECK_INTERVAL:-300}
SERVER_HOST_API_URL: ${SERVER_HOST_API_URL:-http://openmetadata-server:8585/api}
PIPELINE_SERVICE_CLIENT_VERIFY_SSL: ${PIPELINE_SERVICE_CLIENT_VERIFY_SSL:-"no-ssl"}
PIPELINE_SERVICE_CLIENT_SSL_CERT_PATH: ${PIPELINE_SERVICE_CLIENT_SSL_CERT_PATH:-""}
#Database configuration for postgresql
DB_DRIVER_CLASS: ${DB_DRIVER_CLASS:-org.postgresql.Driver}
DB_SCHEME: ${DB_SCHEME:-postgresql}
DB_PARAMS: ${DB_PARAMS:-allowPublicKeyRetrieval=true&useSSL=false&serverTimezone=UTC}
DB_USER: ${DB_USER:-openmetadata_user}
DB_USER_PASSWORD: ${DB_USER_PASSWORD:-openmetadata_password}
DB_HOST: ${DB_HOST:-postgresql}
DB_PORT: ${DB_PORT:-5432}
OM_DATABASE: ${OM_DATABASE:-openmetadata_db}
# ElasticSearch Configurations
ELASTICSEARCH_HOST: ${ELASTICSEARCH_HOST:- elasticsearch}
ELASTICSEARCH_PORT: ${ELASTICSEARCH_PORT:-9200}
ELASTICSEARCH_SCHEME: ${ELASTICSEARCH_SCHEME:-http}
ELASTICSEARCH_USER: ${ELASTICSEARCH_USER:-""}
ELASTICSEARCH_PASSWORD: ${ELASTICSEARCH_PASSWORD:-""}
SEARCH_TYPE: ${SEARCH_TYPE:- "elasticsearch"}
ELASTICSEARCH_TRUST_STORE_PATH: ${ELASTICSEARCH_TRUST_STORE_PATH:-""}
ELASTICSEARCH_TRUST_STORE_PASSWORD: ${ELASTICSEARCH_TRUST_STORE_PASSWORD:-""}
ELASTICSEARCH_CONNECTION_TIMEOUT_SECS: ${ELASTICSEARCH_CONNECTION_TIMEOUT_SECS:-5}
ELASTICSEARCH_SOCKET_TIMEOUT_SECS: ${ELASTICSEARCH_SOCKET_TIMEOUT_SECS:-60}
ELASTICSEARCH_KEEP_ALIVE_TIMEOUT_SECS: ${ELASTICSEARCH_KEEP_ALIVE_TIMEOUT_SECS:-600}
ELASTICSEARCH_BATCH_SIZE: ${ELASTICSEARCH_BATCH_SIZE:-10}
ELASTICSEARCH_PAYLOAD_BYTES_SIZE: ${ELASTICSEARCH_PAYLOAD_BYTES_SIZE:-10485760} #max payLoadSize in Bytes
ELASTICSEARCH_INDEX_MAPPING_LANG: ${ELASTICSEARCH_INDEX_MAPPING_LANG:-EN}
#eventMonitoringConfiguration
EVENT_MONITOR: ${EVENT_MONITOR:-prometheus}
EVENT_MONITOR_BATCH_SIZE: ${EVENT_MONITOR_BATCH_SIZE:-10}
EVENT_MONITOR_PATH_PATTERN: ${EVENT_MONITOR_PATH_PATTERN:-["/api/v1/tables/*", "/api/v1/health-check"]}
EVENT_MONITOR_LATENCY: ${EVENT_MONITOR_LATENCY:-[]}
#pipelineServiceClientConfiguration
PIPELINE_SERVICE_CLIENT_ENABLED: ${PIPELINE_SERVICE_CLIENT_ENABLED:-true}
PIPELINE_SERVICE_CLIENT_CLASS_NAME: ${PIPELINE_SERVICE_CLIENT_CLASS_NAME:-"org.openmetadata.service.clients.pipeline.airflow.AirflowRESTClient"}
PIPELINE_SERVICE_IP_INFO_ENABLED: ${PIPELINE_SERVICE_IP_INFO_ENABLED:-false}
PIPELINE_SERVICE_CLIENT_HOST_IP: ${PIPELINE_SERVICE_CLIENT_HOST_IP:-""}
PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER: ${PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER:-"noop"}
#airflow parameters
AIRFLOW_USERNAME: ${AIRFLOW_USERNAME:-admin}
AIRFLOW_PASSWORD: ${AIRFLOW_PASSWORD:-admin}
AIRFLOW_TIMEOUT: ${AIRFLOW_TIMEOUT:-10}
AIRFLOW_TRUST_STORE_PATH: ${AIRFLOW_TRUST_STORE_PATH:-""}
AIRFLOW_TRUST_STORE_PASSWORD: ${AIRFLOW_TRUST_STORE_PASSWORD:-""}
FERNET_KEY: ${FERNET_KEY:-jJ/9sz0g0OHxsfxOoSfdFdmk3ysNmPRnH3TUAbz3IHA=}
#secretsManagerConfiguration
SECRET_MANAGER: ${SECRET_MANAGER:-db}
#parameters:
OM_SM_REGION: ${OM_SM_REGION:-""}
OM_SM_ACCESS_KEY_ID: ${OM_SM_ACCESS_KEY_ID:-""}
OM_SM_ACCESS_KEY: ${OM_SM_ACCESS_KEY:-""}
#email configuration:
OM_EMAIL_ENTITY: ${OM_EMAIL_ENTITY:-"OpenMetadata"}
OM_SUPPORT_URL: ${OM_SUPPORT_URL:-"https://slack.open-metadata.org"}
AUTHORIZER_ENABLE_SMTP : ${AUTHORIZER_ENABLE_SMTP:-false}
OPENMETADATA_SERVER_URL: ${OPENMETADATA_SERVER_URL:-""}
OPENMETADATA_SMTP_SENDER_MAIL: ${OPENMETADATA_SMTP_SENDER_MAIL:-""}
SMTP_SERVER_ENDPOINT: ${SMTP_SERVER_ENDPOINT:-""}
SMTP_SERVER_PORT: ${SMTP_SERVER_PORT:-""}
SMTP_SERVER_USERNAME: ${SMTP_SERVER_USERNAME:-""}
SMTP_SERVER_PWD: ${SMTP_SERVER_PWD:-""}
SMTP_SERVER_STRATEGY: ${SMTP_SERVER_STRATEGY:-"SMTP_TLS"}
# Heap OPTS Configurations
OPENMETADATA_HEAP_OPTS: ${OPENMETADATA_HEAP_OPTS:--Xmx1G -Xms1G}
# Mask passwords values in UI
MASK_PASSWORDS_API: ${MASK_PASSWORDS_API:-false}
#OpenMetadata Web Configuration
WEB_CONF_URI_PATH: ${WEB_CONF_URI_PATH:-"/api"}
#HSTS
WEB_CONF_HSTS_ENABLED: ${WEB_CONF_HSTS_ENABLED:-false}
WEB_CONF_HSTS_MAX_AGE: ${WEB_CONF_HSTS_MAX_AGE:-"365 days"}
WEB_CONF_HSTS_INCLUDE_SUBDOMAINS: ${WEB_CONF_HSTS_INCLUDE_SUBDOMAINS:-"true"}
WEB_CONF_HSTS_PRELOAD: ${WEB_CONF_HSTS_PRELOAD:-"true"}
#Frame Options
WEB_CONF_FRAME_OPTION_ENABLED: ${WEB_CONF_FRAME_OPTION_ENABLED:-false}
WEB_CONF_FRAME_OPTION: ${WEB_CONF_FRAME_OPTION:-"SAMEORIGIN"}
WEB_CONF_FRAME_ORIGIN: ${WEB_CONF_FRAME_ORIGIN:-""}
#Content Type
WEB_CONF_CONTENT_TYPE_OPTIONS_ENABLED: ${WEB_CONF_CONTENT_TYPE_OPTIONS_ENABLED:-false}
#XSS-Protection
WEB_CONF_XSS_PROTECTION_ENABLED: ${WEB_CONF_XSS_PROTECTION_ENABLED:-false}
WEB_CONF_XSS_PROTECTION_ON: ${WEB_CONF_XSS_PROTECTION_ON:-true}
WEB_CONF_XSS_PROTECTION_BLOCK: ${WEB_CONF_XSS_PROTECTION_BLOCK:-true}
#CSP
WEB_CONF_XSS_CSP_ENABLED: ${WEB_CONF_XSS_CSP_ENABLED:-false}
WEB_CONF_XSS_CSP_POLICY: ${WEB_CONF_XSS_CSP_POLICY:-"default-src 'self'"}
WEB_CONF_XSS_CSP_REPORT_ONLY_POLICY: ${WEB_CONF_XSS_CSP_REPORT_ONLY_POLICY:-""}
#Cache
WEB_CONF_CACHE_CONTROL: ${WEB_CONF_CACHE_CONTROL:-""}
WEB_CONF_PRAGMA: ${WEB_CONF_PRAGMA:-""}
expose:
- 8585
- 8586
ports:
- "8585:8585"
- "8586:8586"
depends_on:
elasticsearch:
condition: service_healthy
postgresql:
condition: service_healthy
execute-migrate-all:
condition: service_completed_successfully
networks:
- app_net
healthcheck:
test: [ "CMD", "wget", "-q", "--spider", "http://localhost:8586/healthcheck" ]
ingestion:
container_name: openmetadata_ingestion
image: docker.getcollate.io/openmetadata/ingestion:1.4.3
depends_on:
elasticsearch:
condition: service_started
postgresql:
condition: service_healthy
openmetadata-server:
condition: service_started
environment:
AIRFLOW__API__AUTH_BACKENDS: "airflow.api.auth.backend.basic_auth,airflow.api.auth.backend.session"
AIRFLOW__CORE__EXECUTOR: LocalExecutor
AIRFLOW__OPENMETADATA_AIRFLOW_APIS__DAG_GENERATED_CONFIGS: "/opt/airflow/dag_generated_configs"
DB_HOST: ${AIRFLOW_DB_HOST:-postgresql}
DB_PORT: ${AIRFLOW_DB_PORT:-5432}
AIRFLOW_DB: ${AIRFLOW_DB:-airflow_db}
DB_USER: ${AIRFLOW_DB_USER:-airflow_user}
DB_SCHEME: ${AIRFLOW_DB_SCHEME:-postgresql+psycopg2}
DB_PASSWORD: ${AIRFLOW_DB_PASSWORD:-airflow_pass}
# extra connection-string properties for the database
# EXAMPLE
# require SSL (only for Postgres)
# properties: "?sslmode=require"
DB_PROPERTIES: ${AIRFLOW_DB_PROPERTIES:-}
# To test the lineage backend
# AIRFLOW__LINEAGE__BACKEND: airflow_provider_openmetadata.lineage.backend.OpenMetadataLineageBackend
# AIRFLOW__LINEAGE__AIRFLOW_SERVICE_NAME: local_airflow
# AIRFLOW__LINEAGE__OPENMETADATA_API_ENDPOINT: http://openmetadata-server:8585/api
# AIRFLOW__LINEAGE__JWT_TOKEN: ...
entrypoint: /bin/bash
command:
- "/opt/airflow/ingestion_dependency.sh"
expose:
- 8080
ports:
- "8080:8080"
networks:
- app_net
volumes:
- ingestion-volume-dag-airflow:/opt/airflow/dag_generated_configs
- ingestion-volume-dags:/opt/airflow/dags
- ingestion-volume-tmp:/tmp
networks:
app_net:
ipam:
driver: default
config:
- subnet: "172.16.240.0/24"