In this doc, we will cover how to setup a Connection to Kafka.

Prerequisites

  • Access credentials
  • Kafka topic, consumer configs

Create new Kafka Read Connector

Figure 1Figure 1

Figure 1

Figure 2Figure 2

Figure 2

  • Connection Name (required): The name to identify this connection with, such as Kafka Sample Read Connection.

  • Kafka Cluster Identifier (required): Must be unique across the full Ascend environment. It lets Ascend store the data read from a specific Kafka cluster (e.g., dev cluster) into the correct location in Ascend identified by this identifier.

  • Bootstrap Servers (required): This is a comma-separated list of host and port pairs that are the addresses of the Kafka brokers in a "bootstrap" Kafka cluster that a Kafka client connects to initially to bootstrap itself. In other words, it provides the initial hosts that act as the starting point for a Kafka client to discover the full set of alive servers in the cluster. A host and port pair uses : as the separator as following examples:
    localhost:9092
    localhost:9092,another.host:9092

  • Consumer Group Id (optional): Consumers can join a consumer group by using group id. Ascend will assign a default one if this is not passed in.

  • Consumer Configs (optional): Open set of configs in Kaka property style to pass down to the consumer, such as security.protocol=SASL_SSL Learn More

Figure 3Figure 3

Figure 3

Create new credentials if Required Credentials checkbox was selected and you do not want to select from the already created ones. (Figure 3)

Example SASL.JAAS.CONFIG below from SASL Client Config
[SASL Client Config](

sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
    username="alice" \
    password="alice-secret";
Figure 4Figure 4

Figure 4

You can also choose to Edit an existing credentials by clicking the EDIT button next to each credentials. (Figure 4)

Figure 5Figure 5

Figure 5

(Figure 5) Use TEST CONNECTION button check if the connection is a SUCCESS* and then click CREATE AND USE CONNECTION**.

📘

If your Kafka is in a private subnet or VPC and not accessible from the public internet. Please contact [email protected] to set up networking.


Did this page help you?