Demystify New Features of RocketMQ and Its Practices in Financial Scenarios

What Is Request-Reply?

Figure 1.1. Request-Reply mode

Implementation Logic of Request-Reply

The RR call involves three roles: producer, broker, and consumer.

Implementation Logic of Producer

Figure 2.1. Schematic diagram of producer
Figure 2.2. Structure of RequestResponseFuture

Implementation Logic of Consumer

Figure 2.3. Schematic diagram of a consumer

Implementation Logic of Broker

Figure 2.4. Schematic diagram of broker

Request-Reply Practices in Financial Scenarios

Financial businesses require continuous and stable services 24/7. In addition, they require strong fault tolerance to quickly shield node failures, ensure the success rate, and quickly restore. Based on this specific application scenario, WeBank adds multiple features, such as active application redundancy, proximity-based service, and circuit breaker, to build a secure and reliable financial-grade message bus called DeFiBus.

Figure 3.1. The architecture of the bus

Fast Fail-and-Retry

Figure 3.2. Schematic diagram of fast fail-and-retry

Circuit Breaker Mechanism

Figure 3.3. Schematic diagram of circuit breaker

Isolation Mechanism

The queue-level isolation mechanism is mainly used as the circuit breaker mechanism of the producer’s retries and services.

Figure 3.4. Schematic diagram of isolation

Dynamic Scaling of Queues

Dynamic scaling is designed to ensure that the number of queues remains consistent with the number of consumers. Hence, the number of queues consumed by each consumer after load balancing is consistent. When the producer evenly sends data, loads of consumers remain balanced even with different numbers of allocated queues.

Figure 3.5. Schemaic diagrameue scale-out
Figure 3.6. Schematic diagram of queue scale-in

Load Balancing Transition

When the load balancing result changes, the consumer of RocketMQ directly updates the old result to the new result, which is a jump process from A to B. If there are many consumers and brokers, the numbers of consumers and queues obtained by different consumers during load balancing may be inconsistent, resulting in inconsistent load balancing results. When the results are inconsistent, the queues may encounter problems such as missed messages and repeated messages in the listening process. For synchronous call scenarios, when you miss listening to a message in the queue, the message processing of the queue takes a long time or even times out, resulting in a call failure.

Figure 3.7. Schematic diagram of load balancing transition

Active Zone-Redundancy of Applications

To meet the requirements of high availability and disaster recovery, services are deployed in at least two Internet data centers (IDCs). When service in one IDC fails and becomes unavailable, a healthy instance in another IDC automatically takes over the traffic. During deployment, the request end and service end are deployed in both IDCs. When both centers are normal, the request end sends a request to the same IDC according to the principle of proximity-based service. Cross-IDC connections are maintained only through heartbeats. The service provider preferentially listens to the queues in the same IDC during the subscription.

Figure 3.8. Schematic diagram of proper IDC running
Figure 3.9. Schematic diagram of an improper application running
Figure 3.10. Schematic diagram of broker failure

Conclusion

This article mainly introduced the Request-Reply mode, which is a new feature of RocketMQ. In this mode, after messages are sent, the producer waits for the consumer to consume all messages and return a response message, which produces an effect similar to the RPC call. The Request-Reply mode allows RocketMQ to call messages synchronously. On this foundation, developers can develop more new features. To better support financial scenarios, WeBank adds multiple features, such as active application redundancy, proximity-based service, and circuit breaker, in order to build a secure and reliable financial-grade message bus called DeFiBus. At present, most of the achievements have been made open-source by WeBank through DeFiBus. More general practice summaries and achievements in sharding and addressing will be introduced in the future.

Original Source:

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store