5.18.2023 Release Notes

These are the release notes for May 18, 2023

🎁 PREVIEWS 🎁

Data Shares - Private Preview (Gen2 environments only):
- Easy and secure sharing of data products across all supported Data Planes.
- Key enabler of an Ascend for Data Mesh strategy.
- Contact your Ascend representative to request early access.

🚀 FEATURES 🚀

Ascend Hosted Cloud customers on the Enterprise Tier can now create Data Services using the Spark with Iceberg Data Plane (Gen2 environments only).

✨ ENHANCEMENTS ✨

Ascend Clusters can now be moved between compute nodes when they are idle (no jobs processing), which makes it possible to optimize how workloads are packed, potentially onto fewer compute nodes.
- This can result in cost savings and better compute utilization.
Listing of source partitions for BigQuery Read Connectors has been optimized when the source object is a BigQuery View.
Change references to the Spark Data Plane type in the UI to "Spark with Iceberg" for consistency.

🔧 BUGFIXES 🔧

All environments (Gen1/Gen2):
- Fix case where Read Connectors occassionally failed to display one of their existing partitions in the Partitions tab of the UI.
Gen1 environments:
- Resolve an issue on Gen1 environments which triggered a "No space left on device" error caused by incorrectly sizing (too small) ephemeral storage size volumes for Ascend Clusters.
Gen2 environments:
- All Data Planes:
  - Enable updating components with failed Data Quality tests.
    - Users received an exception when trying to update a Data Quality check from "Error" to "Warn" behavior, when the Data Quality check had previously failed - this is resolved.
  - Fix a bug where users were unable to use the "Test Connection" button for creating a new connection when a Data Plane Configuration had not been set yet.
  - Fix a race condition that could cause missing data when the schema changes on a Transform with multiple partitions.
  - Fix a typo/spelling error for the Data Service Type header in the Site Admin, Site Configuration section.
- Snowflake Data Plane:
  - Raise error when schema contains a column with data type DECIMAL(0,0) that cannot be converted round-trip.
    - This addresses a behavior which was causing unintended, repeated table re-creation in Snowflake.
- BigQuery Data Plane:
  - Fix handling of "My Tables" ingest when the source object contains a dash ("-") character.
  - When creating a Transform downstream from a non-materialized Read Connector (created with "My Tables"), which uses a large number of sharded tables as a source, the generated query now supports a larger number of sharded tables in the source.
    - Address the error "400 Resources exceeded during query execution: Not enough resources for query planning - too many subqueries or query is too complex".
  - Change behavior to suppress the harmless "409 Already Exists" error which would appear randomly.