8.26.2021 Release Notes

🚀 FEATURES🚀

• MS SQL Server Connection (read + write) uses the newly released MS SQL Spark connector to boost performance
• Delta Lake write connection

✨ ENHANCEMENTS✨

• Polish test connection error messages for S3 and GCS
• Only use whole or half cores for Spark and pin requests and limits
• Snowflake write connections add a new option to keep column case; this option defaults to convert the column names to upper case per Snowflake's convention
• Turn on MySQL rewriteBatchedStatements to improve write performance to MySQL 4 - 40x
• Date partitions in BigQuery now support column types of Date and DateTime in addition to Timestamp
• Polish error message when user supplied schema does not match schema from the source (adding more detail for the nested schema case)
• Mongo CDC connector can resume from cold start by switching from resume_token to the oplog timestamps
• Switch MongoDB Read Connector to ingest with the user-defined explicit schema and avoid auto inference from the collection (which can lead to an inconsistent schema)
• Update Azure blob Go library to v0.13.0
• Increase limits for image pulls in EKS
• UI improvements:
  Autocomplete column names in SQL editor for tables without aliases
  Improve error message with empty schema on S3 Read Connector
  Update PySpark template to build transforms more easily

🔧 BUGFIXES🔧

• Google Sheets Read Connector fixes for handling the "snapshot" configuration
• Delete Azure blobs in 256 sub batches
• Stabilize queries from the Ascend UI by downgrading Spark version
• Fix MySQL non-SSL JDBC connection to not show SSL errors
• Fix BigQuery Write Connection conflicting user-provided staging credentials with system credentials in cases of Ascend GCP deployments
• Fix Spark SQL making the page unresponsive when converting "{{" syntax
• Fix partition display not consistent when components are paused
• Fix partition switching does not load new limit
• Fix Intercom button overlapping other UI elements
• Fix double X to close panel on Create Data Service and Create Data Flow panels
• Fix Export button in Build/Browse panel
• Fix intermittent loading of the connection/connector information on existing connectors