Amazon Kinesis

Prerequisites

• Access credentials
• Data stream on AWS Kinesis

Create new Amazon Kinesis Connection

Create new Amazon Kinesis Data Stream connection

In Figure 3:

• Connection Name (required): The name to identify this connection, such as Kinesis Sample Read Connection.
• Region Name (required): The region name, such as us-west-1.
• Optional IAM Role ARN: Optional IAM role to assume.
• Requires Credentials (required): Automatically checked, as credentials are required for Kinesis Data Streams
• Choose Credentials (required): Credentials to use for connecting to AWS Kinesis. You can choose to create new credentials or use/edit existing AWS credentials.

Creating Credentials

If you chose to create new credentials, you will need to provide:

• Credential Name (required): The name to identify this credential.
• Credential Type (required): Automatically selected to be Amazon Kinesis Data Streams.
• AWS Access Key ID (required): The AWS Access Key associated with your AWS account.
• AWS Secret Access Key (required): The Secret Key associated with the above mentioned Access Key.

Check out our blog post to learn more about credentials in Ascend.

You can also choose to edit existing credentials by clicking the EDIT button next to each credential. (Figure 5)

In order to establish a connection and read in records from a Kinesis Data Stream, the IAM user must have access to perform the following actions on the data stream:

• kinesis:GetShardIterator
• kinesis:GetRecords
• kinesis:ListStreams
• kinesis:ListShards
• kinesis:DescribeStreamSummary
  Here's an example policy configuration for an IAM user to create a read connector for the data stream kinesis:us-west-1:1234567890:stream/StockTradeData:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "kinesis:GetShardIterator",
                "kinesis:GetRecords"
            ],
            "Resource": "arn:aws:kinesis:us-west-1:1234567890:stream/StockTradeData"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "kinesis:ListStreams",
                "kinesis:ListShards"
            ],
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": "kinesis:DescribeStreamSummary",
            "Resource": "arn:aws:kinesis:us-west-1:1234567890:stream/StockTradeData"
        }
    ]
}

Testing and Creating the Connection

(Figure 6) Use TEST CONNECTION button to check whether all S3 permissions are correctly configured. If ALL PASS, click CREATE.