7.22.2021 Release Notes

🚀 FEATURES🚀

• Nested type support for Read Connectors in the UI
• Input partition configuration for SparkSQL, PySpark, and Scala transforms (available under Advanced Settigns). The partition configuration allows for adding in metadata columns as well as filtering down input partitions for non-primary inputs.

✨ ENHANCEMENTS✨

• Spark python to 3.9
• Replace "Blocked" state with "Waiting" when a component's upstream is still running schema inference
• Include the error message when losing a Spark worker
• SDK: Downloading Dataflow + Data Service templates sort Dict keys to keep ordering the same between different downloads

🔧 BUGFIXES🔧

• Improve Loki query cache to avoid "Connection Refused" and "Remote End Disconnected" errors in Read Connectors