yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-08-31 12:39:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update multithreading docs (#16513)

Browse Source
This commit is contained in:
IceS2 2024-06-03 14:52:24 +02:00 committed by GitHub
parent 7558ccc432
commit 1c48a317c7
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 15 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
openmetadata-docs/content/partials/v1.4/connectors/yaml/database/source-config-def.md
View File

@ -12,4 +12,12 @@ The `sourceConfig` is defined [here](https://github.com/open-metadata/OpenMetada
**databaseFilterPattern**, **schemaFilterPattern**, **tableFilterPattern**: Note that the filter supports regex as include or exclude. You can find examples [here](/connectors/ingestion/workflows/metadata/filter-patterns/database) **databaseFilterPattern**, **schemaFilterPattern**, **tableFilterPattern**: Note that the filter supports regex as include or exclude. You can find examples [here](/connectors/ingestion/workflows/metadata/filter-patterns/database)
{% /codeInfo %} **threads (beta)**: The number of threads to use when extracting the metadata using multithreading. Please take a look [here](/connectors/ingestion/workflows/metadata/multithreading) before configuring this.
**incremental (beta)**: Incremental Extraction configuration. Currently implemented for:
- [BigQuery](/connectors/ingestion/workflows/metadata/incremental-extraction/bigquery)
- [Redshift](/connectors/ingestion/workflows/metadata/incremental-extraction/redshift)
- [Snowflake](/connectors/ingestion/workflows/metadata/incremental-extraction/snowflake)
{% /codeInfo %}

7
openmetadata-docs/content/partials/v1.4/connectors/yaml/database/source-config.md
View File

@ -5,6 +5,11 @@
markDeletedTables: true markDeletedTables: true
includeTables: true includeTables: true
includeViews: true includeViews: true
# threads: 4
# incremental:
# enabled: true
# lookbackDays: 7
# safetyMarginDays: 1
# includeTags: true # includeTags: true
# databaseFilterPattern: # databaseFilterPattern:
# includes: # includes:
@ -27,4 +32,4 @@
# excludes: # excludes:
# - table3 # - table3
# - table4 # - table4
``` ```