Kafka Cluster with Docker Compose

Apache Kafka is described itself as “distributed commit log” or more recently “distributing streaming platform”. Kafka brokers can work together in a group to provide distributed environment which called as cluster.

Spring Boot KafkaTestContainer

In event based architecture integration test is hard. Because of event based architecture is asynchronous, it is hard to test that events are sent and handled by consumers. Another problem is, need to run Kafka when application started for testing. In some mocking/embedded solutions are restrict the Kafka’s features so it is not flexible to write tests with these mocking/embedded solutions.

Spring Boot Kafka

Apache Kafka is a distributed messaging system which is initially developed by Jay Kreps when he was working in Linkedin. Apache Kafka designed as a distributed system which helps to scale horizontally. It is using in few different use cases as data pipeline between different systems/microservices, storage or powering logging/monitoring for the high traffic application. In this blogpost, project simulates a basic product data pipeline of e-commerce site.

Spring Boot with Docker

Containers have been making revolutionary changes in software world. Previous systems were mostly powered by virtual machines. Today many software pioneers like Google, Amazon, Redhat and Docker Inc. convert the industry in that way. Docker was released in 2013 by Solomon Hykes. Now it is developing by Docker Inc. It is relying on LXC(Linux Containers) which was invented in 2008 by IBM.

Introduction to Testcontainers

Spring Boot makes it easy to setup and start a microservice project. But in microservice’s world, it is hard to test changes in all architecture. Because it is hard to setup same environment in local. So each service should responsible their own changes and guarantee that it won’t break any RPC call or business logic with new changes. Integration tests help in that manner to release the changes in more confident. So it is important to create an environment which is as much as close to production.