Integrating Spring Boot with Apache Kafka

Event-driven architectures let services communicate asynchronously and scale independently. In this article we wire Spring Boot to Apache Kafka end to end.

Introduction

Kafka is a distributed commit log. Spring for Apache Kafka gives us idiomatic producers and consumers backed by templates and listener containers.

Configuring the Producer

Define a ProducerFactory and a KafkaTemplate:

@Bean
public KafkaTemplate<String, OrderEvent> kafkaTemplate(
    ProducerFactory<String, OrderEvent> pf) {
  return new KafkaTemplate<>(pf);
}

Consuming Messages

Annotate a method with @KafkaListener:

@KafkaListener(topics = "orders", groupId = "billing")
public void onOrder(OrderEvent event) {
  log.info("Received {}", event.id());
}

Error Handling and Retries

Use a DefaultErrorHandler with a dead-letter topic so poison messages don't block the partition.