4.28.2022 Release Notes

Here are the release notes for April 28, 2022.

✨ ENHANCEMENTS✨

• Scale Spark driver resources relative to total number of partitions and number of files in order to improve spark job startup time for large jobs
• Increase field size limit for Salesforce bulk loads in order to handle larger max field sizes
• Upgrade Parquet library in Spark 3.2.0 runtime to handle Integer overflow bug
• Release SDK 0.2.37 (Adds support for setting Data Plane configuration per Data Service)

🔧 BUGFIXES🔧

• Fix usage of connection overrides in Snowflake connections
• Fix Reformat SQL behavior in Query editor for Data Plane environments to use the right SQL dialect for the Data Plane
• Delta Lake WC: Fix handling of no records in upstream component
• Delta Lake WC: Fix max partitionsToDelete error by batching partition delete requests