9.23.2021 Release Notes

🚀 FEATURES🚀

Announcing Data SHAs!
Data SHAs is a new feature of Ascend in which the platform will create a SHA for all the output artifacts generated from your Ascend pipelines. By collecting this SHA, Ascend can stop downstream propagation if the partitions are unchanged even if the upstream's code and / or partitions have changed. This week's release starts collecting the Data SHAs on a go-forward basis and we will be beginning the backfill process for historical Data SHAs. This process (and how aggressively we backfill) is configurable -- your Ascend account representative will be reaching out to help coordinate. After the historical Data SHAs are collected, your Ascend environment will be able to start leveraging them to avoid processing / computation when the data artifacts are in fact the same.

✨ ENHANCEMENTS✨

Support Excel as a parser option for Blob Stores and SFTP
Redshift Read Connector supports reading through an "UNLOAD" command to S3
Redshift Write Connector supports writing through a "COPY" command to S3
Add ability to register Delta Lake Tables on S3 to the AWS Glue Catalog
Add custom JDBC connection options for MySQL to customize connection as necessary
When applying a component via the SDK, if there exists a component with the same ID of a different type, first try to delete it to support a changed component type (SDK version 0.2.29)
Enable filtering partitions tab by partition state (Running, Error, Up To Date)

🔧 BUGFIXES🔧

Fix Records Preview in "Snapshot" mode for Google Sheets
Remove Spark classloader logs from partition logs to decrease log noise
Fix an issue with updating credentials on the Site Admin screen showing "found more than 1 credential with same id"
Fix Records Preview in DynamoDB sometimes timing out
Create a record loss event for MongoDB CDC Streams when the stream needs to recover from a CDC Oplog Retention Failure
Fix Write Connectors UI sometimes not loading the details (like "table name")
Fix Date type in MySQL CDC stream rendering