We made the following changes to help calculating data sets quality score (refer to this great article for more info):
We’ve added two new columns, elementary_test_results.failed_row_count and dbt_tests.quality_dimension. This data enables you to gain deeper insights into data quality and better track your data pipeline’s health.
We automatically fill these fields for common test (dbt, dbt_utils, and dbt_expectations), if you would like them to work for other test, or overwrite the default ones, you will need to fill the test’s meta with:
quality_dimension - Name of the quality dimension this test measures.
failed_row_count_calc - An SQL expression to calculate the amount of failed rows from the test (count(*), sum(n_records), etc).
Thank you Saurabh Jha for working with us on this!
This feature is still experimental, and is not reflected in the report nor the alerts.
Alert Suppression in CLI 🚦
Now, you can suppress alerts in the Command Line Interface (CLI) using the —suppression-interval flag.
This feature gives you more control over your alerts and helps you manage notifications effectively.
We’ve added a new option to order Slack messages by test run time. This makes it easier to track test results and prioritize action items - Thanks Willi Mueller for suggesting!
