Amazon S3

In this doc, we will cover how to setup a Connection to an Amazon S3 bucket.

Prerequisites

  • Access credentials
  • Data location on AWS S3
  • Data Schema (column names and column type)

Create new S3 connection

952952

Figure 1

613613

Figure 2

In Figure 2:

  • Bucket (optional): The bucket name, such as ascend-io-sample-read-data.
  • Connection Name (required): The name to identify this connection with, such as Amazon S3 Sample Read S3.
  • S3 API Endpoint (optional): This is for endpoint which is not S3 but S3 compatible.
  • S3 Region (optional): The region name, such as us-west-2.
  • Required Credentials (Optional): If de-selected, only the publicly accessible buckets can be accessed.
  • Choose Credentials (Required): Credentials to use for connecting to S3 if Required Credentials checkbox is selected.
700700

Figure 3

Create new credentials if Required Credentials checkbox was selected and you do not want to select from the already created ones. (Figure 3)

613613

Figure 4

You can also choose to Edit an existing credentials by clicking the EDIT button next to each credentials. (Figure 4)

Here's an example S3 policy configuration for this IAM user to create a read connector for data within the bucket s3://ascend-io-playground-bucket/:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation",
                "s3:GetBucketPolicy",
                "s3:GetBucketAcl",
                "s3:GetObject",
                "s3:GetObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::ascend-io-playground-bucket",
                "arn:aws:s3:::ascend-io-playground-bucket/*"
            ]
        }
    ]
}

Testing and Creating the Connection

509509

Figure 5

(Figure 5) Use TEST CONNECTION button to check whether all S3 permissions are correctly configured If SUCCESS, click CREATE AND USE CONNECTION.