yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-08-31 12:39:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

Added alation conn arg docs (#15495)

Browse Source
This commit is contained in:
Onkar Ravgan 2024-03-08 18:40:04 +05:30 committed by GitHub
parent 226db58818
commit 1f8e232487
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 20 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
openmetadata-docs/content/v1.3.x/connectors/metadata/alation/yaml.md
View File

@ -127,6 +127,20 @@ The access token created using the steps mentioned [here](https://developer.alat
{% /codeInfo %}
{% codeInfo srNumber=21 %}
**connectionArguments**: These are additional parameters for Alation. If not specified the ingestion will use the predefined pagination logic.
The following arguments are intended to be used in conjunction and are specifically for Alation DataSource APIs:
- skip: This parameter determines the count of records to bypass at the start of the dataset. When set to 0, as in this case, it means that no records will be bypassed. If set to 10, it will bypass the first 10 records.
- limit: This argument specifies the maximum number of records to return. Here, it's set to 10, meaning only the first 10 records will be returned.
To perform incremental ingestion, these arguments should be used together. For instance, if there are a total of 30 datasources in Alation, the ingestion can be configured to execute three times, with each execution ingesting 10 datasources.
- 1st execution: {"skip": 0, "limit": 10}
- 2nd execution: {"skip": 10, "limit": 10}
- 3rd execution: {"skip": 20, "limit": 10}
{% /codeInfo %}
#### Sink Configuration
{% codeInfo srNumber=20 %}
@ -176,6 +190,12 @@ source:
```yaml {% srNumber=18 %}
alationTagClassificationName: alationTags
```
```yaml {% srNumber=21 %}
connectionArguments: {
"skip": 0,
"limit": 10,
}
```
```yaml {% srNumber=19 %}
sourceConfig:
config: