Amazon S3
In this doc, we will cover how to setup a Connection to an Amazon S3 bucket.
Prerequisites
- Access credentials
- Data location on AWS S3
- Data Schema (column names and column type)
Create new S3 connection
![Screen Shot 2021-03-04 at 4.53.37 AM.png 952](https://files.readme.io/c8d2d31-Screen_Shot_2021-03-04_at_4.53.37_AM.png)
Figure 1
![Screen Shot 2021-03-04 at 8.15.52 AM.png 613](https://files.readme.io/d963417-Screen_Shot_2021-03-04_at_8.15.52_AM.png)
Figure 2
In Figure 2:
- Bucket (optional): The bucket name, such as
ascend-io-sample-read-data
. - Connection Name (required): The name to identify this connection with, such as
Amazon S3 Sample Read S3
. - S3 API Endpoint (optional): This is for endpoint which is not S3 but S3 compatible.
- S3 Region (optional): The region name, such as
us-west-2
. - Required Credentials (Optional): If de-selected, only the publicly accessible buckets can be accessed.
- Choose Credentials (Required): Credentials to use for connecting to S3 if Required Credentials checkbox is selected.
![Screen Shot 2021-03-04 at 4.09.39 AM.png 700](https://files.readme.io/59ca0f5-Screen_Shot_2021-03-04_at_4.09.39_AM.png)
Figure 3
Create new credentials if Required Credentials checkbox was selected and you do not want to select from the already created ones. (Figure 3)
![Screen Shot 2021-03-04 at 8.15.52 AM.png 613](https://files.readme.io/b70195d-Screen_Shot_2021-03-04_at_8.15.52_AM.png)
Figure 4
You can also choose to Edit an existing credentials by clicking the EDIT button next to each credentials. (Figure 4)
Here's an example S3 policy configuration for this IAM user to create a read connector for data within the bucket s3://ascend-io-playground-bucket/:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation",
"s3:GetBucketPolicy",
"s3:GetBucketAcl",
"s3:GetObject",
"s3:GetObjectAcl"
],
"Resource": [
"arn:aws:s3:::ascend-io-playground-bucket",
"arn:aws:s3:::ascend-io-playground-bucket/*"
]
}
]
}
Testing and Creating the Connection
![Screen Shot 2021-03-04 at 8.22.04 AM.png 509](https://files.readme.io/b9820f0-Screen_Shot_2021-03-04_at_8.22.04_AM.png)
Figure 5
(Figure 5) Use TEST CONNECTION button to check whether all S3 permissions are correctly configured If SUCCESS, click CREATE AND USE CONNECTION.
Updated 6 months ago