Developer HubDocsSDKFree Trial
Back to All

3.24.2022 Release Notes

about 2 years ago

Here are the release notes for March 24, 2022.

:sparkles: ENHANCEMENTS:sparkles:

  • Surface Spark node capacity issues as unschedulable error rather than a runtime error
  • Improve records preview performance by caching column names
  • Big Query Write Connector: Avoid "Too many sources provided" errors in BigQuery write connection by coalescing write dataframe to a max of 10,000 URIs
  • Big Query Connection: Support Pagination
  • Big Query Connection: Improve performance of schema generation for large tables
  • Big Query Data Plane: Add support for query cancellation
  • Improve error message when spark worker driver pods cannot connect to kubernetes master. It will say 'unable to connect to kubernetes master', rather than 'spark worker failed to connect', or other similar messages.

:wrench: BUGFIXES:wrench:

  • Fix component log collection in Azure by handling changes in AKS node taints
  • Redshift Write Connector: Fix redshift alter table failures for columns with reserved keywords by escaping column names
  • Fix pre/post sql Unread result found error by fetching from the cursor before closing it
  • Fix issue where the pause state of a component appeared to change when the component was updated (it didn't change, and now we reflect that correctly)
  • Salesforce Read Connector: Fix bug where if the first and last modified time for new data are the same (typically a single-record update), the data wouldn't be ingested. Now the data will be ingested as expected
  • Salesforce Read Connector: Always fetch records with include_deleted enabled in order to catch changes from record deletion
  • Salesforce Read Connector: Improve performance of bulk load for small data sizes
  • Salesforce Read Connector: Fix handling of all non-compound field types during bulk load