Connectors

Messaging & Event Streaming

Apache Kafka

Distributed event streaming: produce and consume messages on topics with consumer groups
Apache Kafka API reference
Signs in with
Connection details
Runs on
Cloud or On-Prem Worker
Operations
9
Ready-made triggers
0

Connect Apache Kafka

  1. In Neblex open Connect › Connections and click New Connection.
  2. Search for Apache Kafka and give the connection a clear name.
  3. Fill in the fields. Secret values are encrypted and never shown again.
  4. Click Test, then Create Connection.
If the system is only reachable inside your network, choose an On-Prem Worker under Test from and run the flow on that Worker. See On-Prem Workers.
FieldWhat to enter
Bootstrap brokersRequiredComma-separated host:port list, e.g. broker1:9092,broker2:9092.
Use TLSOptionalTurning this off sends traffic and credentials without transport encryption. Use plaintext only for a broker that explicitly requires it.
SASL authenticationRequiredChoose the mechanism the cluster enables. PLAIN sends the password itself, so use it only with TLS.
SASL usernameRequired
SASL passwordRequiredSecret

Use it in a flow

  1. Open a flow, open the node library and search for Apache Kafka.
  2. Drag it onto the canvas and choose an Operation and your connection.
  3. Map the operation's fields from earlier steps, or click Map with Matt.
  4. Use Test this step to see what it returns, then save.

To start a flow from Apache Kafka without a ready-made trigger: a Live stream trigger with Apache Kafka as the stream source starts a run for each message as it arrives (see Streaming triggers).

Operations

9 operations in 3 groups. Each one is a step you can add to a flow.

Connection1
pingping
Records4
Delete recordsdelete_recordsDelete retained records before an offset in one partition.
Fetch recordsfetch_recordsFetch a bounded record page from one partition and return exact keys, values, headers, timestamps, and offsets.
Get partition offsetsget_partition_offsetsGet retained earliest/latest offsets and optionally resolve a timestamp to an offset.
Produce a recordproduceProduce one exact record with broker acknowledgement; omitted partitions use Kafka's keyed Murmur2 selection.
Topics4
Create a topiccreate_topicCreate a topic with a partition count + replication factor.
Delete a topicdelete_topicDelete a topic.
Get topicget_topicGet one topic and its exact partition identifiers.
List topicslist_topicsList topics and their exact partition identifiers with a bounded result count.