02.22.2024 Release Notes
10 months ago
These are the release notes for February 22, 2024
🚀 FEATURES 🚀
- 🕒 Scheduled Trigger Read Connector released to General Availability:
- This Custom Connection Type allows the creation of Scheduled Trigger Read Connector components.
- These components can trigger downstream components in Ascend, on a schedule!
✨ ENHANCEMENTS ✨
- All environments (Gen1/Gen2)
- Add new Change Data Capture, Multi-Table Stream Replication Mode for MySQL Read Connectors:
- This mode allows users to ingest all, or a subset of databases/schemas and tables from MySQL in a single connector, as opposed to having a single table per connector.
- The "File has header row" setting is now enabled by default for Read Connectors that offer the CSV parser as an option.
- It is now easier to select multiple columns/attributes and remove them in one click in the Generate Schema UI on Read Connectors.
- Add Clustering Field option in the BigQuery Write Connector:
- Ascend will now create/maintain that clustering field on tables it creates in the downstream BigQuery dataset.
- Add support for static filtering expression on BigQuery, Clickhouse, CockroachDB, DB2, DB2AS400, Databricks, Hive, Ingres, MSSQL, Maria, MySQL, Oracle, Progress DB, Redshift, Snowflake, Synapse, and Vertica Read Connectors:
- Allows the use of a static filter expression which is added to the SQL WHERE clause.
- Add new Change Data Capture, Multi-Table Stream Replication Mode for MySQL Read Connectors:
- Gen2 environments
- Enable Ascend Cluster-level installation of custom Python packages via pip in Cluster Management. If using custom Ascend Cluster container images, update your images to the latest base Ascend image to use this feature:
- Simplifies use cases where customers previously installed pip packages in Custom Python Read Connections and/or Connectors.
- We encourage customers to migrate to the new cluster-level method as soon as possible.
- We may deprecate the previous Connection/Connector pip install method in favor of the new method, and will notify customers if we do so.
- Simplifies use cases where customers previously installed pip packages in Custom Python Read Connections and/or Connectors.
- Enable Ascend Cluster-level installation of custom Python packages via pip in Cluster Management. If using custom Ascend Cluster container images, update your images to the latest base Ascend image to use this feature:
🔧 BUGFIXES 🔧
- All environments (Gen1/Gen2)
- Fix bug causing intermittent "use of closed network connection" errors.
- Fix bug causing components to become unpaused after editing/saving them in the UI.
- Fix expand and collapse (full-screen mode) for SQL editor fields on Merge Transform components.
- Prevent Ascend SDK
dry_runfrom failing when applying a dataflow against an empty dataflow in an Ascend environment:- The purpose of
dry_runin this case is to list the expected changes that applying the code would cause.
- The purpose of
- Gen2 environments
- Prevent deletion of Ascend Cluster pools that are in use by a Data Service or Dataflow:
- To delete, first, disassociate the cluster pool you wish to delete from any Data Services/Dataflows where it is in use.
- Spark with Iceberg Data Plane
- Fix an internal statistics collection bug encountered in the engine when no column is available to collect statistics.
- All Data Planes
- Fix a case where old partitions were not cleaned up properly when users edit a component, save the changes, and then undo the changes back to the original state.
- Fix bug that caused Delta Lake Write Connectors to fail when using the AWS Glue Catalog.
- Prevent deletion of Ascend Cluster pools that are in use by a Data Service or Dataflow: