yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-11-02 03:39:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(observe): update docs for remote executor, databricks (#10393)

Browse Source
This commit is contained in:
Mayuri Nehate 2024-04-29 18:07:46 +05:30 committed by GitHub
parent 3ab4ec9b44
commit 8e5c0dc2d6
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 15 additions and 15 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/managed-datahub/observe/assertions.md
View File

@ -1,7 +1,7 @@
# Assertions
:::note Contract Monitoring Support
Currently we support Snowflake, Databricks, Redshift, and BigQuery for out-of-the-box contract monitoring as part of Acryl Observe.
Currently we support Snowflake, Redshift, BigQuery, and Databricks for out-of-the-box contract monitoring as part of Acryl Observe.
:::
An assertion is **a data quality test that finds data that violates a specified rule.**

6
docs/managed-datahub/observe/column-assertions.md
View File

@ -18,7 +18,7 @@ import FeatureAvailability from '@site/src/components/FeatureAvailability';
Can you remember a time when an important warehouse table column changed dramatically, with little or no notice? Perhaps the number of null values suddenly spiked, or a new value was added to a fixed set of possible values. If the answer is yes, how did you initially find out? We'll take a guess - someone looking at an internal reporting dashboard or worse, a user using your your product, sounded an alarm when a number looked a bit out of the ordinary.
There are many reasons why important columns in your Snowflake, Redshift, or BigQuery tables may change - application code bugs, new feature rollouts, etc. Oftentimes, these changes break important assumptions made about the data used in building key downstream data products like reporting dashboards or data-driven product features.
There are many reasons why important columns in your Snowflake, Redshift, BigQuery, or Databricks tables may change - application code bugs, new feature rollouts, etc. Oftentimes, these changes break important assumptions made about the data used in building key downstream data products like reporting dashboards or data-driven product features.
What if you could reduce the time to detect these incidents, so that the people responsible for the data were made aware of data issues before anyone else? With Acryl DataHub Column Assertions, you can.
@ -41,7 +41,7 @@ Note that an Ingestion Source _must_ be configured with the data platform of you
Acryl DataHub's **Ingestion** tab.
> Note that Column Assertions are not yet supported if you are connecting to your warehouse
> using the DataHub CLI or a Remote Ingestion Executor.
> using the DataHub CLI.
## What is a Column Assertion?
@ -121,7 +121,7 @@ another always-increasing number - that can be used to find the "new rows" that
`Edit Assertions` and `Edit Monitors` privileges for the entity. This is granted to Entity owners by default.
2. **Data Platform Connection**: In order to create a Column Assertion, you'll need to have an **Ingestion Source**
configured to your Data Platform: Snowflake, BigQuery, or Redshift under the **Ingestion** tab.
configured to your Data Platform: Snowflake, BigQuery, Redshift, or Databricks under the **Ingestion** tab.
Once these are in place, you're ready to create your Column Assertions!

6
docs/managed-datahub/observe/custom-sql-assertions.md
View File

@ -22,7 +22,7 @@ If the answer is yes, how did you find out? We'll take a guess - someone looking
a number looked a bit out of the ordinary. Perhaps your table initially tracked purchases made on your company's e-commerce web store, but suddenly began to include purchases made
through your company's new mobile app.
There are many reasons why an important Table on Snowflake, Redshift, or BigQuery may change in its meaning - application code bugs, new feature rollouts,
There are many reasons why an important Table on Snowflake, Redshift, BigQuery, or Databricks may change in its meaning - application code bugs, new feature rollouts,
changes to key metric definitions, etc. Often times, these changes break important assumptions made about the data used in building key downstream data products
like reporting dashboards or data-driven product features.
@ -49,7 +49,7 @@ Note that an Ingestion Source _must_ be configured with the data platform of you
tab.
> Note that SQL Assertions are not yet supported if you are connecting to your warehouse
> using the DataHub CLI or a Remote Ingestion Executor.
> using the DataHub CLI.
## What is a Custom SQL Assertion?
@ -120,7 +120,7 @@ The **Assertion Description**: This is a human-readable description of the Asser
`Edit Assertions`, `Edit Monitors`, **and the additional `Edit SQL Assertion Monitors`** privileges for the entity. This is granted to Entity owners by default.
2. **Data Platform Connection**: In order to create a Custom SQL Assertion, you'll need to have an **Ingestion Source** configured to your
Data Platform: Snowflake, BigQuery, or Redshift under the **Integrations** tab.
Data Platform: Snowflake, BigQuery, Redshift, or Databricks under the **Integrations** tab.
Once these are in place, you're ready to create your Custom SQL Assertions!

8
docs/managed-datahub/observe/freshness-assertions.md
View File

@ -22,7 +22,7 @@ months without being updated with fresh data?
Perhaps a bug had been introduced into an upstream Airflow DAG
or worse, the person in charge of maintaining the Table has departed from your organization entirely.
There are many reasons why an important Table on Snowflake, Redshift, or BigQuery may fail to be updated as often as expected.
There are many reasons why an important Table on Snowflake, Redshift, BigQuery, or Databricks may fail to be updated as often as expected.
What if you could reduce the time to detect these incidents, so that the people responsible for the data were made aware of data
issues _before_ anyone else? What if you could communicate commitments about the freshness or change frequency
@ -49,7 +49,7 @@ Note that an Ingestion Source _must_ be configured with the data platform of you
tab.
> Note that Freshness Assertions are not yet supported if you are connecting to your warehouse
> using the DataHub CLI or a Remote Ingestion Executor.
> using the DataHub CLI.
## What is a Freshness Assertion?
@ -147,7 +147,7 @@ Freshness Assertions also have an off switch: they can be started or stopped at
`Edit Assertions` and `Edit Monitors` privileges for the entity. This is granted to Entity owners by default.
2. **Data Platform Connection**: In order to create a Freshness Assertion, you'll need to have an **Ingestion Source** configured to your
Data Platform: Snowflake, BigQuery, or Redshift under the **Integrations** tab.
Data Platform: Snowflake, BigQuery, Redshift, or Databricks under the **Integrations** tab.
Once these are in place, you're ready to create your Freshness Assertions!
@ -260,7 +260,7 @@ As part of the **Acryl Observe** module, Acryl DataHub also provides **Smart Ass
dynamic, AI-powered Freshness Assertions that you can use to monitor the freshness of important warehouse Tables, without
requiring any manual setup.
If Acryl DataHub is able to detect a pattern in the change frequency of a Snowflake, Redshift, or BigQuery Table, you'll find
If Acryl DataHub is able to detect a pattern in the change frequency of a Snowflake, Redshift, BigQuery, or Databricks Table, you'll find
a recommended Smart Assertion under the `Validations` tab on the Table profile page:
<p align="center">

8
docs/managed-datahub/observe/volume-assertions.md
View File

@ -22,7 +22,7 @@ If the answer is yes, how did you find out? We'll take a guess - someone looking
a number looked a bit out of the ordinary. Perhaps your table initially tracked purchases made on your company's e-commerce web store, but suddenly began to include purchases made
through your company's new mobile app.
There are many reasons why an important Table on Snowflake, Redshift, or BigQuery may change in its meaning - application code bugs, new feature rollouts,
There are many reasons why an important Table on Snowflake, Redshift, BigQuery, or Databricks may change in its meaning - application code bugs, new feature rollouts,
changes to key metric definitions, etc. Often times, these changes break important assumptions made about the data used in building key downstream data products
like reporting dashboards or data-driven product features.
@ -50,7 +50,7 @@ Note that an Ingestion Source _must_ be configured with the data platform of you
tab.
> Note that Volume Assertions are not yet supported if you are connecting to your warehouse
> using the DataHub CLI or a Remote Ingestion Executor.
> using the DataHub CLI.
## What is a Volume Assertion?
@ -140,7 +140,7 @@ Volume Assertions also have an off switch: they can be started or stopped at any
`Edit Assertions` and `Edit Monitors` privileges for the entity. This is granted to Entity owners by default.
2. **Data Platform Connection**: In order to create a Volume Assertion, you'll need to have an **Ingestion Source** configured to your
Data Platform: Snowflake, BigQuery, or Redshift under the **Integrations** tab.
Data Platform: Snowflake, BigQuery, Redshift, or Databricks under the **Integrations** tab.
Once these are in place, you're ready to create your Volume Assertions!
@ -238,7 +238,7 @@ As part of the **Acryl Observe** module, Acryl DataHub also provides **Smart Ass
dynamic, AI-powered Volume Assertions that you can use to monitor the volume of important warehouse Tables, without
requiring any manual setup.
If Acryl DataHub is able to detect a pattern in the volume of a Snowflake, Redshift, or BigQuery Table, you'll find
If Acryl DataHub is able to detect a pattern in the volume of a Snowflake, Redshift, BigQuery, or Databricks Table, you'll find
a recommended Smart Assertion under the `Validations` tab on the Table profile page:
<p align="center">