09.20.2023 Release Notes

These are the release notes for September 20, 2023

📰 NEWS 📰

• 🚀 Ascend for Data Mesh is now enabled by default for all new customers & new environments!
  • This technology enables customers to run data pipelines across multiple data planes within a single Ascend Gen2 environment (1 Data Plane per Data Service).
  • This includes the Data Shares feature, which is now enabled for all new customers/environments by default going forward!
    • Data Shares enable easy and secure sharing of data products across all supported Data Planes.
  • If you are an existing customer on Gen2 and do not already have these features enabled, please contact Ascend Support to discuss enabling them for your environment.
  • Start building your Data Mesh today!
• 📧 Ascend Email Notifications are now available to all customers!
  • This builds on our existing Platform Notifications (previously Webhooks only) to allow you to set up good old email notifications.
  • Email notifications are sent from a "no-reply" address and sender name "Ascend.io Notification".

✨ FEATURES ✨

• All environments (Gen1/Gen2)
  • Add new BigQuery Write Connector Write Strategies
    • Partition Write Using Ascend Partition Id: Ascend partitions guide BigQuery table writes. New partitions are added, obsolete ones in the target are removed, and changed partitions undergo an automatic "partition swap".
    • Partition Append: With partition field and type set, Ascend sets up partitioned BigQuery tables. New partitions are filtered to contain only rows with values surpassing the previously written largest partition field value.
    • Partition Merge: With partition field and type set, Ascend uses partitioned BigQuery tables and compares partition IDs. New partitions are added from Ascend, and if Ascend partitions have mismatched record counts with the partitions in the target table, they are overwritten.
  • Add new Ascend SDK download templates (TEMPLATES_V3) to make source control of downloaded Data Services simpler and cleaner.
    • 😎 Stay tuned for more on this feature as we release a new version of the Ascend CLI - this will be a game changer for simplifying CI/CD with Ascend!

✨ ENHANCEMENTS ✨

• All environments (Gen1/Gen2)
  • Add a new option in Refresh Schedule of Read Connector to reset all the errors when a new refresh is triggered.
  • Improve handling of large error messages to ensure they display correctly in the UI.
    • Previously, users might encounter a "gRPC message size exceeded limit" error instead.

🔧 BUGFIXES 🔧

• All environments (Gen1/Gen2)
  • Fix a JSON parsing error that caused no Job Usage data to show up in the Observe UI.
• Ascend Cloud
  • Fix Microsoft OAuth login for customers on the Ascend Cloud.
• Gen2 environments
  • All Data Planes
    • Fix a bug that was causing the listing phase of Read Connectors to run for much longer than usual.
    • Fix an issue where manually configured Ascend Cluster pools in the Data Service -> Data Plane settings weren't recognized as active, leading to task failures.
      • These Data Service level 'overrides' are now properly recognized as active Ascend Clusters, preventing premature shutdowns.
    • Prevent task failure due to "409 Already Exists" error for Data Planes by handling cases where users made configuration changes and renamed a component at the same time.
  • Databricks Data Plane
    • Fix inability to create Write Connectors downstream of a Merge Transform.