Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

38 lines
3.1 KiB
Markdown
Raw Normal View History

---
2025-08-12 13:29:42 +05:30
title: databaseServiceQueryLineagePipeline
slug: /main-concepts/metadata-standard/schemas/metadataingestion/databaseservicequerylineagepipeline
---
# DatabaseServiceQueryLineagePipeline
*DatabaseService Query Lineage Pipeline Configuration.*
## Properties
2025-08-12 13:29:42 +05:30
- **`type`**: Pipeline type. Refer to *#/definitions/databaseLineageConfigType*. Default: `DatabaseLineage`.
- **`queryLogDuration`** *(integer)*: Configuration to tune how far we want to look back in query logs to process lineage data. Default: `1`.
- **`queryLogFilePath`** *(string)*: Configuration to set the file path for query logs.
- **`resultLimit`** *(integer)*: Configuration to set the limit for query logs. Default: `1000`.
- **`parsingTimeoutLimit`** *(integer)*: Configuration to set the timeout for parsing the query in seconds. Default: `300`.
- **`filterCondition`** *(string)*: Configuration the condition to filter the query history.
2025-08-12 13:29:42 +05:30
- **`schemaFilterPattern`**: Regex to only fetch tables or databases that matches the pattern. Refer to *../type/filterPattern.json#/definitions/filterPattern*.
- **`tableFilterPattern`**: Regex exclude tables or databases that matches the pattern. Refer to *../type/filterPattern.json#/definitions/filterPattern*.
- **`databaseFilterPattern`**: Regex to only fetch databases that matches the pattern. Refer to *../type/filterPattern.json#/definitions/filterPattern*.
- **`storedProcedureFilterPattern`**: Regex to only fetch stored procedures that matches the pattern. Refer to *../type/filterPattern.json#/definitions/filterPattern*.
- **`overrideViewLineage`** *(boolean)*: Set the 'Override View Lineage' toggle to control whether to override the existing view lineage. Default: `False`.
- **`processViewLineage`** *(boolean)*: Set the 'Process View Lineage' toggle to control whether to process view lineage. Default: `True`.
- **`processQueryLineage`** *(boolean)*: Set the 'Process Query Lineage' toggle to control whether to process query lineage. Default: `True`.
- **`processStoredProcedureLineage`** *(boolean)*: Set the 'Process Stored ProcedureLog Lineage' toggle to control whether to process stored procedure lineage. Default: `True`.
- **`threads`** *(integer)*: Number of Threads to use in order to parallelize lineage ingestion. Minimum: `1`. Default: `1`.
2025-08-12 13:29:42 +05:30
- **`processCrossDatabaseLineage`** *(boolean)*: Set the 'Process Cross Database Lineage' toggle to control whether to process table lineage across different databases. Default: `False`.
- **`crossDatabaseServiceNames`** *(array)*: Set 'Cross Database Service Names' to process lineage with the database.
- **Items** *(string)*
2025-08-12 13:29:42 +05:30
- **`enableTempTableLineage`** *(boolean)*: Handle Lineage for Snowflake Temporary and Transient Tables. . Default: `False`.
- **`incrementalLineageProcessing`** *(boolean)*: Set the 'Incremental Lineage Processing' toggle to control whether to process lineage incrementally. Default: `True`.
## Definitions
2025-08-12 13:29:42 +05:30
- **`databaseLineageConfigType`** *(string)*: Database Source Config Usage Pipeline type. Must be one of: `['DatabaseLineage']`. Default: `DatabaseLineage`.
2025-08-12 13:29:42 +05:30
Documentation file automatically generated at 2025-08-12 05:39:47.683420+00:00.