Kafka Overview

There are conflicting opinions about microservices, where some people straight refuse to use microservices, as they are considered tech debt. However, most of the companies I worked with try to contain certain logic or domains within a separate service. An ideal microservice is fully isolated from other microservices and does nothing effectively. However, we create them to perform a function, and to trigger the function we must communicate to the service what we want it to do. In web development, normally we communicate via HTTP requests, however, have shortcomings:

• They aren’t always reliable, especially if the communicating applications tend to be unreliable, too
• It is hard to communicate a large array of data or organize the processing of data spread in time
• Multiple microservices communicate with each other at the same time, being dependencies of each other are hard to control, let alone restore the pipeline if one of them fails

Kafka Software Architecture Role

Given its high level of durability but with an unknown latency, Kafka is the best choice in applications where processing takes a significant amount of time from the consumer’s perspective.