8.31.2021 Release Notes
over 3 years ago
🚀 FEATURES🚀
- New Connection Type called “Ascend System Usage”. Creating this connection in your Ascend environment will allow for accessing three raw data streams: Node, Pod, and Jobs. The “Node” stream is a log of what nodes (instances) exist in the environment at minute granularity. The “Pod” stream is a log of what pods are running on the nodes at minute granularity. The “Job” stream is a log of all the jobs that Ascend has scheduled, when it was scheduled, completed, and the resources consumed. Some customers may have previously had a “DFU stream” enabled by Ascend, which these streams serve to replace in a unified way whilst enabling self-service.
✨ ENHANCEMENTS✨
- Delete parent directories during cleanup in Azure Blob Store
- SDK: Add in preview_connection function that returns the predicted schema and some sample records for a connection. This can be used to bootstrap the creation of programmatic connections or as part of scripts to re-sync schemas according to their sources.
- Retry tasks on transient Azure auth errors
- Enable semantic IDs for connections and credentials. These ID’s now work the same way as Data Service, Dataflow, and Component IDs where re-naming in the UI changes the “slug” format that you can use to refer to these, which also makes it simpler to deploy code from one Data Service to another without having to change references. With this change, existing references to connections / credentials through UUID will continue to work. With the latest SDK (0.2.27), you can now also use these semantic IDs.
- UI: default new transforms to a “Running” state to avoid needing to explicitly select between “Running” and “Paused”
- Hide Requires Credentials checkbox unless Connection Type is selected in Site Admin form
- Allow an empty dataset to process through a “Repartition” transform as a single partition rather than raising an error that repartitioning is not possible on an empty data set.
- Order Connection Browser items by name
- UI: Add “Edit Component” button to the context menu (the menu displayed when right clicking a component or clicking the 3 “dots” in the upper right of a component)
- UI: Add “Refresh Now” button to the context menu (the menu displayed when right clicking a component or clicking the 3 “dots” in the upper right of a component)
🔧 BUGFIXES🔧
- Improve stability and responsiveness of Ascend API by decreasing network connection hangs (enabling smoother SDK workflows)
- Fix case in SFTP Read Connection where files and directories would appear in connection browser and then not be ingested
- Sync Dataflow view in UI across browsers to ensure that deleted components are not shown (allowing both users and browsers to see the same state)