Lesson
Kafka Replication means having multiple copies of the data, spread across multiple servers/brokers. This helps in maintaining high availability in case one of the brokers goes down and is unavailable to serve the requests. Data Replication helps prevent data loss by writing the same data to more than one broker. In Kafka, replication means that data is written down not just to one broker, but many. The replication factor is a topic setting and is specified at topic creation time. This replication factor is configured at the topic level, and the unit of replication is the topic partition. In Apache Kafka, producers and consumers exchange messages in the form of key-value pairs. When working with Kafka, it's essential to serialize the data into bytes before sending it to Kafka and deserialize it back into its original format when consuming messages. Kafka allows us to use custom serializers and deserializers for keys and values even though Publisher API provides serializers like IntegerSerializer, StringSerializer etc, same sense of deserializer. The serializer is used by the message publisher while deserializer is used by the message consumer. In short-form it refers as Kafka SerDe Apache Kafka consumers are applications that read data from Kafka topics and process it. In Apache Kafka, a consumer offset is a critical concept that helps keep track of a consumer's progress in reading messages from a Kafka topic. Applications that read data from Kafka topics are known as consumers. Applications integrate a Kafka client library to read from Apache Kafka. Excellent client libraries exist for almost all programming languages that are popular today including Python, Java, Go, and others. Apache Kafka's consumer properties are configuration settings that control the behavior and functionality of Kafka consumers. Apache Kafka's producer properties are configuration settings that control the behavior and functionality of Kafka producers. These properties are specified when configuring a Kafka producer in your application. Apache Kafka's broker properties are configuration settings that control the behavior and functionality of individual Kafka brokers within a Kafka cluster. To consume messages from an Apache Kafka topic using Java, you'll need to use the Kafka Consumer API provided by the Apache Kafka library.