5.26.2022 Release Notes

Here are the release notes for May 26, 2022.

🚀 FEATURES 🚀

• New Databricks Data Plane

✨ ENHANCEMENTS ✨

• Add support for setting Data Plane configuration per Dataflow
• Use the same Spark autoscaling ratio regardless of the max cluster scale limit in order to avoid under-provisioning Spark resources in resource-constrained configurations resulting in OOMs

🔧 BUGFIXES 🔧

• Fix bug in Delta Lake on Azure Read Connector that prevented data ingestion with the incremental read strategy with manifest file
• SDK Release 0.2.41: V2 Data Service template will automatically discover all Dataflows in its subdirectory (previously this was not working correctly)
• SDK Release 0.2.42: Fix Decimal field type support to default to the same precision and scale settings as Spark