Kafka
In this doc, we will cover how to setup a Connection to Kafka.
Prerequisites
- Access credentials
- Kafka topic, consumer configs
Create new Kafka Read Connector

Figure 1

Figure 2
-
Connection Name (required): The name to identify this connection with, such as
Kafka Sample Read Connection
. -
Kafka Cluster Identifier (required): Must be unique across the full Ascend environment. It lets Ascend store the data read from a specific Kafka cluster (e.g., dev cluster) into the correct location in Ascend identified by this identifier.
-
Bootstrap Servers (required): This is a comma-separated list of host and port pairs that are the addresses of the Kafka brokers in a "bootstrap" Kafka cluster that a Kafka client connects to initially to bootstrap itself. In other words, it provides the initial hosts that act as the starting point for a Kafka client to discover the full set of alive servers in the cluster. A host and port pair uses : as the separator as following examples:
localhost:9092
localhost:9092,another.host:9092 -
Consumer Group Id (optional): Consumers can join a consumer group by using group id. Ascend will assign a default one if this is not passed in.
-
Consumer Configs (optional): Open set of configs in Kaka property style to pass down to the consumer, such as
security.protocol=SASL_SSL
Learn More

Figure 3
Create new credentials if Required Credentials checkbox was selected and you do not want to select from the already created ones. (Figure 3)
Example SASL.JAAS.CONFIG below from SASL Client Config
[SASL Client Config](
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
username="alice" \
password="alice-secret";

Figure 4
You can also choose to Edit an existing credentials by clicking the EDIT button next to each credentials. (Figure 4)

Figure 5
(Figure 5) Use TEST CONNECTION button check if the connection is a SUCCESS* and then click CREATE AND USE CONNECTION**.
If your Kafka is in a private subnet or VPC and not accessible from the public internet. Please contact [email protected] to set up networking.
Updated about 1 year ago