Apache Spark
Reading data into Spark using File-Based Access
Ascend's File-Based Access enables Spark systems to scalably retrieve records from any Component in a Dataflow.
Please review the File-Based Access basics to gain familiarity with the URI format and generate credentials.
Then, with a valid Spark session, we can set the s3a library to point to Ascend and read in data.
Read Ascend Data to a Spark DataFrame
Open Recipe
Updated 9 months ago