File-Based Access

Ascend's File-Based Access is a way to retrieve the records from any Component in Ascend as Snappy-compressed Parquet files.

Retrieving records through this file-based interface enables:

• High performance
• Bulk retrieval
• Well-known interface that many tools are already suitable to integrate with (such as Spark processing engines and notebooks)

S3-Compatible API

Regardless of which cloud provider Ascend has been deployed to, the File-Based Access provides the endpoint with S3 compatibility, transparently serving up files from the underlying cloud's object store.

All environments use the endpoint https://s3.ascend.io.

The S3 Protocol URI follows the organization within Ascend:

s3://<Ascend Subdomain>/<Data Service Id>/<Dataflow ID>/<Component ID>

Each component in the UI has a section called 'Data Integrations' which includes the full S3 Protocol URI.

The API authenticates with an Ascend Access Key ID + Secret pair. Generate these credentials either through a Service Account or a Developer Access Key.

Using the AWS S3 CLI

Because Ascend exposes as an S3 endpoint, you can use most any existing S3 client, including Amazon's aws s3 Command Line Interface (AWS CLI).

1. Create an Access Key ID + Secret pair in Ascend.
   You can use either a Service Account or a Developer Access Key.
2. Save the Access Key ID and Secret in your ~/.aws/credentials file, where the AWS CLI can find them:

[trial]
aws_access_key_id=YOUR_KEY_ID
aws_secret_access_key=YOUR_SECRET_ACCESS_KEY

📘

Set up a separate profile for using Ascend

Because the Ascend S3 server is not the same as the Amazon AWS S3 server, we recommend you use a separate profile so that you can continue to use the [default] profile with Amazon AWS S3 server. In this example, we set up a profile called [trial].

3. Set the AWS CLI addressing style to virtual by adding the following configuration to ~/.aws/config. The Ascend S3 endpoint must be accessed using Virtual-Hosted Style Requests.

[profile trial]
source_profile = trial
s3 =
    addressing_style = virtual

4. Make a S3 requests (such a ls, cp, etc) through the aws s3 CLI to access any read connector, transform, or data feed in Ascend. Specify the --endpoint_url and the profile for each request.

> aws s3 ls --profile trial --endpoint-url https://s3.ascend.io s3://trial/Getting_Started_with_Ascend/IoT_Device_and_Weather_Analysis/K_Means_Cluster/