Configuration

Overview

In the configuration, our applications behave as consumers that communicate with each other to complete processes. They use a JSON format to send and receive messages. Consumer groups are used to scale our applications and share the same identifier. For example, when two model controllers coexist, each one is a consumer and they both belong to the same consumer group.

A topic is the data structure Kafka uses to store and distribute messages. Topics organize messages in partitions. Consumers are attached to different topics. When a topic receives a message, it sends it to a consumer group. Just one of the group’s consumers receives the message. The consumer array contains all connections receiving messages from a topic. Learn more about topic configuration here.

The groups object

{
  "teamState": "",
  "teamId": "",
  "consumer": []
}
ParameterTypeDescription
teamStatestringThe team’s status. Values can be Stable (the consumers are ready to receive messages), Dead (a consumer has an error), Empty (it doesn’t have any consumers), PreparingRebalance (it’s distributing partitions to new consumers), CompletingRebalance (it’s completing the distribution), and Unknown.
teamIdstringThe team’s identifier.
consumerarrayAn array of consumers.

The consumers object

{
  "topicPartitions": [],
  "consumerId": "",
  "clientId": ""
}
ParameterTypeDescription
topicPartitionsarrayAn array of topic partition objects.
consumerIdstringThe consumer’s identifier. Applications may have a different consumers for each instance.
clientIdstringThe client application’s identifier.

The topics object

{
  "consumerApplication": "",
  "producedBy": "",
  "topic": "",
  "description": "",
  "data": {}
}
ParameterTypeDescription
consumerApplicationstringThe application that consumes the topic’s messages.
producedBystringThe application that produces the topic’s messages.
topicstringThe topic’s name.
descriptionstringThe consumer’s action after it receives the message.
dataobjectThe team’s object.

The topics partition object

{
  "topic": "",
  "partition": 0,
  "byteSize": -1,
  "offset": {}
}
ParameterTypeDescription
topicstringThe topic’s identifier.
partitionnumberThe partition’s index number.
byteSizenumberThe disk space used by the partition in bytes. Its value is -1 if the partition has never used its disk space.
offsetobjectThe offset object.

The offset object

{
  "currentOfConsumer": 44562,
  "lastOfTopic": 44562,
  "lag": 0
}
ParameterTypeDescription
currentOfConsumernumberThe last consumed message’s index.
lastOfTopicnumberThe last message index available in the partition.
lagnumberThe difference between the last consumed message’s index and the newest message’s index for a consumer in the topic.