Assertions

Contract Monitoring Support

Currently we support Snowflake, Databricks, Redshift, and BigQuery for out-of-the-box contract monitoring as part of Acryl Observe.

An assertion is a data quality test that finds data that violates a specified rule. Assertions serve as the building blocks of Data Contracts – this is how we verify the contract is met.

How to Create and Run Assertions

Data quality tests (a.k.a. assertions) can be created and run by Acryl or ingested from a 3rd party tool.

Acryl Observe

For Acryl-provided assertion runners, we can deploy an agent in your environment to hit your sources and DataHub. Acryl Observe offers out-of-the-box evaluation of the following kinds of assertions:

• Freshness (SLAs)
• Volume
• Custom SQL
• Column

These can be defined through the DataHub API or the UI.

3rd Party Runners

You can integrate 3rd party tools as follows:

• DBT Test
• Great Expectations

If you opt for a 3rd party tool, it will be your responsibility to ensure the assertions are run based on the Data Contract spec stored in DataHub. With 3rd party runners, you can get the Assertion Change events by subscribing to our Kafka topic using the DataHub Actions Framework.

Alerts

Beyond the ability to see the results of the assertion checks (and history of the results) both on the physical asset’s page in the DataHub UI and as the result of DataHub API calls, you can also get notified via slack messages (DMs or to a team channel) based on your subscription to an assertion change event. In the future, we’ll also provide the ability to subscribe directly to contracts.

With Acryl Observe, you can get the Assertion Change event by getting API events via AWS EventBridge (the availability and simplicity of setup of each solution dependent on your current Acryl setup – chat with your Acryl representative to learn more).

Cost

We provide a plethora of ways to run your assertions, aiming to allow you to use the cheapest possible means to do so and/or the most accurate means to do so, depending on your use case. For example, for Freshness (SLA) assertions, it is relatively cheap to use either their Audit Log or Information Schema as a means to run freshness checks, and we support both of those as well as Last Modified Column, High Watermark Column, and DataHub Operation (see the docs for more details).