09.29.2022 Release Notes

Here are the release notes for Sep 29, 2022.

:rocket: FEATURES :rocket:

  • Support for Machine Learning libraries in Snowflake's Snowpark for Python and Ascend PySpark (scikit-learn and other packages)
  • πŸŽ‰ New: Data Quality Checks πŸŽ‰
    • Run standard checks (UNIQUE, NOT NULL, >, >=, <, <=, IN()) by column name
    • Run custom checks with SQL statements
    • Full-component, and per-partition optimized checks
    • Customizable Error and Warn thresholds
    • Snowflake and Databricks Data Planes only... BigQuery Data Plane coming soon!
    • Reach out to Ascend to enable the feature for your data plane environment.

:sparkles: ENHANCEMENTS :sparkles:

  • Incremental and Snapshot replication strategies for Databricks connections

:wrench: BUGFIXES :wrench:

  • BigQuery Read Connector now performs full refresh in table view
  • Fix bug in relisting logic for Adaptive Source Refresh where certain paths in the source would be missed in source refresh
  • Fix Observe service date ranges and CPU hours in timeline views