Azure Event Hubs
In this doc, we will cover how to setup a Connection to Azure Event Hubs.
Prerequisites
- Event Hubs namespace in Azure as shown below. Make sure to select the "Standard" pricing tier as the "Basic" pricing tier does not have Kafka enabled and therefore will not work with Ascend:
Figure 1
The resource group we created before is called "yyy." The name for our Namespace is AscendConnect. The pricing tier we have chosen is "Standard." Azure Event Hubs has 3 pricing tiers as shown in Figure 2. The Ascend Event Hubs connection is built upon Apache Kafka and therefore you MUST be using either the "Standard" or dedicated Event Hubs tier to connect with Ascend.
Figure 2
Create an Azure Event Hub Read Connector
Next, go to Ascend and select the button to Create a New Read Connector:
Figure 3
After selecting the Read Connector type as "Azure Event Hubs," you will need to fill out the form to create a new connection.
Figure 4
The Event Hubs Namespace Hostname will be the host name that is given to you in Azure (it is not just the hostname but also has ".servicebus.windows.net" appended to it; you can copy it directly from Azure. The identifier is optional. However, if you do include an identifier, it must be unique and not the same as the Hostname.
Create a new credential in Ascend as shown in the Figure 5 below. Add a name for your connection. Your connection string is more complicated to enter so carefully follow the instructions written below.
Figure 5
You can create any credential name you want.
For the Connection String, copy the "Connection string–primary key" from Azure into Ascend. In Azure, on the far left pane in your "Event Hubs Namespace," there is a button for "Shared Access Policies." (Make sure you are checking for Shared Access Policies in your Event Hubs Namespace, not your Event Hubs instance which is housed inside of your Events Hubs Namespace.) Click Shared Access Policies button and then click the first policy there which should be called "RootManagedSharedAccessKey." Copy the third field from the top which is for "Connection string–primary key."
Figure 6
Your connection string primary key from Azure will take the format of:
Endpoint=sb://< UNIQUE EVENT HUBS NAMESPACE HOSTNAME>/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=<UNIQUE SHARED ACCESS KEY>=
You will need to add the following string to the beginning of the connection string given by Azure:
org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="
So your final key will look like:
org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="Endpoint=sb://< UNIQUE EVENT HUBS NAMESPACEHOSTNAME>/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=<UNIQUE SHARED ACCESS KEY>=
Paste the complete connection string and then create the credential and then test it.
When we test our connection, we see that it works successfully:
Figure 7
Connecting to Data in the Event Hub
Follow this link and create a Python script to send data to your event hub (you will need to
install the right dependencies to get the script to run and then input your connection string and event hub name).
https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-python-get-started-send
Updated over 1 year ago