Amazon S3

In this doc, we will cover how to setup a Connection to an Amazon S3 bucket.

Prerequisites

• Access credentials
• Data location on AWS S3
• Data Schema (column names and column type)

Create new S3 connection

In Figure 2:

• Bucket (optional): The bucket name, such as ascend-io-sample-read-data.
• Connection Name (required): The name to identify this connection with, such as Amazon S3 Sample Read S3.
• S3 API Endpoint (optional): This is for endpoint which is not S3 but S3 compatible.
• S3 Region (optional): The region name, such as us-west-2.
• Required Credentials (Optional): If de-selected, only the publicly accessible buckets can be accessed.
• Choose Credentials (Required): Credentials to use for connecting to S3 if Required Credentials checkbox is selected.

Create new credentials if Required Credentials checkbox was selected and you do not want to select from the already created ones. (Figure 3)

You can also choose to Edit an existing credentials by clicking the EDIT button next to each credentials. (Figure 4)

Here's an example S3 policy configuration for this IAM user to create a read connector for data within the bucket s3://ascend-io-playground-bucket/:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation",
                "s3:GetBucketPolicy",
                "s3:GetBucketAcl",
                "s3:GetObject",
                "s3:GetObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::ascend-io-playground-bucket",
                "arn:aws:s3:::ascend-io-playground-bucket/*"
            ]
        }
    ]
}

Testing and Creating the Connection

(Figure 5) Use TEST CONNECTION button to check whether all S3 permissions are correctly configured If SUCCESS, click CREATE AND USE CONNECTION.