Kafka vs. Apache RocketMQ™- Multiple Topic Stress Test Results
Introduction
This post delves into how we will simulate the real scenarios below:
- Message sending and subscription must be concomitant.
- Multiple subscription ends supported to subscribe only to messages you are interested in.
The focus of this post is on Kafka and Apache RocketMQ™, since they both are high-performance applications. Thus, we will target these two products to see which one of them is better in the above scenarios. Before we begin, let’s clarify two concepts first:
What is a Topic?
Topic is an important concept in message-oriented middleware. Every topic represents a message category. With multiple topics, we can categorize and isolate messages.
You can refer to the feeding models in a zoo in the figure below. Every animal can only consume the specific corresponding food.
What is a Partition?
Both Kafka and Apache RocketMQ™ adopt message queues on disks. For the same consumption group, a partition only supports message consumption by one consumption thread. Too few partitions may cause the consumption speed to lag far behind the message generation speed. In actual production environments, a topic will be set in the multi-partition mode to support multiple consumers, as shown in the figure below:
Many topics and environments exist in an actual production environment of internet enterprises. Such production environment requires the message-oriented middleware to guarantee the service stability in the coexistence of multiple topics. Let’s move on to the testing step to check the handling capacities of Kafka and Apache RocketMQ™ on multiple topics when message sending and subscription ends coexist.
Testing Objective
We will compare the impact of the number of topics on the performances of Kafka and Apache RocketMQ™ when sending and receiving ends coexist. In this case, we adopt eight partitions. However, in this stress test, we only pay attention to the performance indicators of the service end. Therefore, the “Exit” criteria for the stress test are as given below.
Keep increasing the stress on the sending end until the system throughput does not increase and the response takes longer. At this time, a performance bottleneck emerges on the service end. Obtain the corresponding optimal throughput of the system, ensuring that there are no messages accumulated throughout the process.
Test scenarios
By default, every topic has eight partitions; every topic matches one subscriber, and the number of topics is increased gradually. The data obtained is as follows:
We can see that no matter how many topics there are, both Kafka and Apache RocketMQ™ can maintain flat TPS on the sending and receiving ends. Simply put, no message accumulation occurs.
Based on the changes in number of topics, we can draw the curves of message handling capacities of Kafka and Apache RocketMQ™, as shown in the figure below:
From the figure above we can see that:
- When the number of topics increases from 64 to 256, the throughput of Kafka dropped by 98.37%.
- When the number of topics increases from 64 to 256, the throughput of Apache RocketMQ™ dropped by only 16%.
Why the huge difference? The difference is attributable to the fact that every topic and partition of Kafka correspond to one physical file. When the number of topics increases, the policy of deconcentrated storage of messages to disks will lead to disk IO competition to cause performance bottlenecks. In contrast, all the messages in Apache RocketMQ™are stored in the same physical file. The number of topics and partitions is just a logic division for Apache RocketMQ™. So the increasing number of topics won’t generate a huge impact on the Apache RocketMQ™ performance.
Test conclusion
When the message sending and consumption ends coexist, the increasing number of topics will cause a drastic decline of Kafka’s throughput, while Apache RocketMQ™ delivers a stable performance. Therefore, Kafka is more suitable for business scenarios with only a few topics and consumption ends, while Apache RocketMQ™ is a better choice for business scenarios with multiple topics and consumption ends.
Appendix:
Test environment
The service end is deployed in the standalone mode. The server configurations are as follows:
Application version:
Test scripts
Summary
In the test above, we saw how Apache RocketMQ™ almost overwhelms Kafka. The result is hardly surprising because Apache RocketMQ™ is born to target internet production requirements. Readers now should understand why Apache RocketMQ™ manages to support the massive messaging services of Alibaba group.
The stress test of multiple-topic scenarios involved in the test only lasted 20 minute. Too short an execution period for a message-oriented middleware product, hence inadequate to identify its stability.
Reference:
