09.29.2022 Release Notes

Here are the release notes for Sep 29, 2022.

🚀 FEATURES 🚀

• Support for Machine Learning libraries in Snowflake's Snowpark for Python and Ascend PySpark (scikit-learn and other packages)
• 🎉 New: Data Quality Checks 🎉
  • Run standard checks (UNIQUE, NOT NULL, >, >=, <, <=, IN()) by column name
  • Run custom checks with SQL statements
  • Full-component, and per-partition optimized checks
  • Customizable Error and Warn thresholds
  • Snowflake and Databricks Data Planes only... BigQuery Data Plane coming soon!
  • Reach out to Ascend to enable the feature for your data plane environment.

✨ ENHANCEMENTS ✨

• Incremental and Snapshot replication strategies for Databricks connections

🔧 BUGFIXES 🔧

• BigQuery Read Connector now performs full refresh in table view
• Fix bug in relisting logic for Adaptive Source Refresh where certain paths in the source would be missed in source refresh
• Fix Observe service date ranges and CPU hours in timeline views