2.3.2022 Release Notes
almost 3 years ago
Here are the release notes for February 3, 2022.
🚀 FEATURES🚀
- Release Spark 3.2.0 as a Spark runtime option in Ascend
- New DynamoDB Write Connector
✨ ENHANCEMENTS✨
- Switch Excel parsing from a Spark library to Pandas for increased maintainability and more relevant configuration options
- Upgrade parquet library to 1.11.2
- Add a record limit for Postgres CDC Read Connectors to ensure that a partition doesn't run out of memory while reading the CDC stream. The default limit is set to 1,000,000 records and we will further increase this as we roll out staging in Postgres CDC to blob store.
- Improve performance of Object Aggregation by loading multiple input files at once
🔧 BUGFIXES 🔧
- Fix bug causing component paused state not being included in transform definition when using SDK
- Fix a case where recursive transforms with the same data may get stuck in "Ready" state