Azure Event Hubs

In this doc, we will cover how to setup a Connection to Azure Event Hubs.

Prerequisites

  • Event Hubs namespace in Azure as shown below. Make sure to select the "Standard" pricing tier as the "Basic" pricing tier does not have Kafka enabled and therefore will not work with Ascend:
2760

Figure 1

The resource group we created before is called "yyy." The name for our Namespace is AscendConnect. The pricing tier we have chosen is "Standard." Azure Event Hubs has 3 pricing tiers as shown in Figure 2. The Ascend Event Hubs connection is built upon Apache Kafka and therefore you MUST be using either the "Standard" or dedicated Event Hubs tier to connect with Ascend.

2598

Figure 2

Create an Azure Event Hub Read Connector

Next, go to Ascend and select the button to Create a New Read Connector:

719

Figure 3

After selecting the Read Connector type as "Azure Event Hubs," you will need to fill out the form to create a new connection.

1992

Figure 4

The Event Hubs Namespace Hostname will be the host name that is given to you in Azure (it is not just the hostname but also has ".servicebus.windows.net" appended to it; you can copy it directly from Azure. The identifier is optional. However, if you do include an identifier, it must be unique and not the same as the Hostname.

Create a new credential in Ascend as shown in the Figure 5 below. Add a name for your connection. Your connection string is more complicated to enter so carefully follow the instructions written below.

1214

Figure 5

You can create any credential name you want.

For the Connection String, copy the "Connection string–primary key" from Azure into Ascend. In Azure, on the far left pane in your "Event Hubs Namespace," there is a button for "Shared Access Policies." (Make sure you are checking for Shared Access Policies in your Event Hubs Namespace, not your Event Hubs instance which is housed inside of your Events Hubs Namespace.) Click Shared Access Policies button and then click the first policy there which should be called "RootManagedSharedAccessKey." Copy the third field from the top which is for "Connection string–primary key."

702

Figure 6

Your connection string primary key from Azure will take the format of:

Endpoint=sb://< UNIQUE EVENT HUBS NAMESPACE HOSTNAME>/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=<UNIQUE SHARED ACCESS KEY>=

You will need to add the following string to the beginning of the connection string given by Azure:

org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="

So your final key will look like:

org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="Endpoint=sb://< UNIQUE EVENT HUBS NAMESPACEHOSTNAME>/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=<UNIQUE SHARED ACCESS KEY>=

Paste the complete connection string and then create the credential and then test it.

When we test our connection, we see that it works successfully:

1988

Figure 7

Connecting to Data in the Event Hub

Follow this link and create a Python script to send data to your event hub (you will need to
install the right dependencies to get the script to run and then input your connection string and event hub name).
https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-python-get-started-send