Kafka Producer and Consumer External Objects

Official Content
This documentation is valid for:

This document describes the steps to send and receive messages through the Publish-Subscribe API of Apache Kafka using some external objects provided for that specific purpose.

The provided External Objects are used to configure parameters, send and receive the messages. The implementation is based on Kafka Consumers.




The consumer can be a daemon that is running all the time waiting for new events. The Consumer method asks the queue or stream for new events and gets a collection of Key-Value.

&Consumer.Configuration = "{'bootstrap.servers': 'localhost:9092', 'group.id': 'mi-nuevo-grupo3', 'auto.offset.reset': 'earliest', 'request.timeout.ms': 500}"
&Timeout = 10000
do while (true)
    &Consumer.Consume(!"Topic1", &Timeout, &ListMessagingResponse)    //&ListMessagingResponse is a Collection of Key, Value, Topic and Error info.


The Producer is asynchronous, which lets to add events to the stream massively. The External Object has a Finish method which waits until all the events are added to the stream.

&Producer.Configuration = "{'bootstrap.servers': 'localhost:9092', 'default.topic.config': {'message.timeout.ms': 10000}}"
do while(true)
    &Text = GetTextToSend()  // &Text is a string with the message that is sent​.
    &OK = &Producer.ProduceAsync("Topic1", "key_1" ,&Text )
    &ListMessagingResponse = &Producer.Finish(1500)  //Waits up to 1.5 seconds for finishing sending messages. 
    if (&ListMessagingResponse.Count <> 1)
        msg("Some error ocurred", status)    

See Also

Appendix - Tips for testing Kafka

1.- Follow the steps of this article

2.- Configure the file kafka<numberversion>/config/server.properties adding the line:
advertised.host.name = <ipmachine>

3.- To test from Windows, you can download http://www.kafkatool.com/