Disaster Tolerance and Remote Multi-Active Regions with MongoShake
On June 29, Alibaba Cloud officially announced the open source MongoShake, a MongoDB-based disaster tolerance and multi-active IDCs/regions solution. MongoShake can synchronize data between data centers with a maximum QPS of 500,000. It solves the inflexible disaster tolerance and single-point write-in problems, implements disaster tolerance across data centers and remote multi-active IDCs/regions, and significantly reduces the O&M workload.
Overview of MongoShake
Mongo-Shake is a universal platform for services based on MongoDB’s oplog. It fetches oplog from source mongo database, and replays in the target mongo database or sends to other ends in different tunnels. If the target side is mongo database, which means replay oplog directly, it’s like a syncing tool that is used to copy data from source MongoDB to another MongoDB to build redundant replication or active-active replication. In addition to this direct way, there are others tunnel types such like rpc, file, tcp, and kafka. Receivers written by users must define their own interfaces to connect to these tunnels respectively. Users can also define their own tunnel type, which is pluggable. When connecting to a third-party message middleware like kafka, the consumer can get the subscriber data in an asynchronous way in pub/sub module flexibly.
MongoShake in Action
MongoShake has shined in various major sales promotions. Take the sales promotion held by Amap on China’s National Day as an example. MongoShake has met Amap’s requirements for disaster tolerance and high availability of multiple data centers and store all data on each of the three data centers. This relieves the pressure of testing and O&M personnel, meets the flexible service switching requirements, and ensures stable service operations under the peak traffic on the National Day.
Meanwhile, as a general platform service, MongoShake also flexibly interconnects closed-loop nodes to adapt to different scenarios, such as log subscription, data center synchronization, and monitoring and auditing. By making MongoShake open source, we hope to enable MongoDB to play a bigger role in the database field.
Disaster Tolerance across Data Centers and Remote Multi-Active IDCs/Regions, Two Tough Issues Resolved by MongoShake
In the current database ecosystem, most systems support data synchronization between multiple node instances, such as MySQL master/slave synchronization and Redis AOF master/slave synchronization. MongoDB even supports replica set synchronization of three nodes or more.
Therefore, data synchronization across units and data centers is especially important at the business layer. To eliminate the current constraints imposed by MongoDB on data synchronization across data centers, Alibaba Cloud develops the MongoShake solution that can be used for inter-instance synchronization or data synchronization across data centers, meeting the disaster tolerance and multi-active IDCs/regions requirements.
Deployment of MongoShake Disaster Tolerance and Remote Multi-Active IDCs/Regions
To further improve the transmission performance, Alibaba Cloud adopts multiple ideas of design, such as multi-end pulling, parallel execution, conflict detection, and asynchronous optimization, to meet developers’ performance requirements. MongoShake also supports DML and DDL synchronization of the database, which ensures data consistency at the bottom layer.
MongoShake’s open source code provides an excellent solution for developers who currently use the MongoDB database. For more information about the details and design of MongoShake as well as the official open source website, visit the following websites:
Reference:
