Azure Data Lake Storage Gen 2

In this doc, we will cover how to setup a Connection to Azure Data Lake Storage Gen 2 (ADLS Gen 2).

Prerequisites

• Microsoft Account
• A Storage Account to use with Azure Data Lake Storage Gen2
• A container inside of your Azure Storage Account

Creating an Azure Date Lake Storage Gen 2 Connection

You will need a Microsoft account to use ADLS. Follow the steps in this link to create an ADLS account: https://docs.microsoft.com/en-us/azure/storage/blobs/create-data-lake-storage-account.

📘

Note: The Storage Account Name must match the Account Name inside your Azure account.

The following steps will get you connected to a data set in ADLS using Ascend.
First, inside your Ascend account, select your Data Service. Here, we have selected "Ascend Doc Service" as our data service. Then, select the BUILD tab on the far left. Under the "CREATE" tab select "READ CONNECTOR."

After selecting the option to create a new READ CONNECTOR, you will encounter a screen prompting for the following fields:

• Connection Name: This is the name that you choose for your connection
• Storage Account Name: The name of the your ALDS storage account resource in Azure
• Container Name: The name of your container in Azure
• Requires Credentials: If de-selected, only the publicly accessible buckets can be accessed.
• Choose Credential: Click 'CHOOSE CREDENTIAL' and then select 'CREATE NEW CREDENTIAL` as shown below and enter credential details as shown in Figure 3

After you click 'Create new credential' you will name it and have the option to select 3 distinct credential types. In this tutorial, we connected Ascend to Azure using a Shared Key which is in the left-side pane of the storage account window in Azure where you can generate a new key and paste it into Ascend. You can also use the Azure AD Service Principal method to create credentials. Documentation for that is here: https://docs.microsoft.com/en-us/azure/storage/common/storage-account-keys-manage?tabs=azure-portal

For "CHOOSE CREDENTIAL," you are going to have to use the credentials from Azure to connect to Ascend.

• Credential Name: This is the name that you choose for your connection
• Credential Type: Since you have already specified "ALDS Gen 2 Blob Storage" as your blob store, this field is pre-selected for you
• Credential Type: The type of credential we want to select is "Azure Shared Key"
• Azure Shared Key: This is a secret key generated in Azure

Testing and Creating the Connection

In Figure 4 above, use TEST CONNECTION button to check whether credentials are correctly configured and If SUCCESS, click CREATE AND USE CONNECTION.