Ingenuity | A New Triumph for X-Engine

Image for post
Image for post

1) What Is X-Engine?

X-Engine is an Alibaba proprietary online transaction processing (OLTP) database storage engine. It is widely used in Alibaba’s core businesses, including DingTalk, core transactions, and Alimama. This engine significantly reduces storage overheads while providing high performance.

2) Hardware Development Trends

In recent years, server-side hardware, including processors and storage, has developed rapidly. In terms of CPUs, Intel’s Xeon CPU family has been upgraded under the first round of the “Process-Architecture-Iteration” model in the 14 nanometer (nm) process. In the 10 nm process, a new Tiger Lake microarchitecture has been released.

Image for post
Image for post
Peak I/O performance of Intel SSDs released over the last decade

3) Compaction in the LSM Tree

In the LSM-tree architecture, write operations are first inserted into the memory. When the usage of the memory reaches the threshold, data is written to the disk and merged with the data in the disk, whose process is similar to that of the merge sort algorithm. The data in the disk is organized hierarchically. When the data at a level reaches the threshold, it is merged with the data at the next lower level, as shown in part (a) of the following figure.

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

4) Overall Architecture

The FPGA programmable acceleration card (PAC) that we use is connected to the server through the PCIe interface. It cannot gain access to the memory and disks as easily as the CPU on the server. In addition, an FPGA has powerful computing capabilities, but the latency of each access is similar to that of an I/O device. These features indicate that we cannot directly apply the preceding CPU-specific compaction algorithm to an FPGA.

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

5) Experimental Evaluation

All the experiments in this article were performed on two Intel Xeon Platinum 8163, 2.5 GHz, 24-core, two-way hyper-threading processors with 512 GB DDR4 2133 memory. The disk contains a RAID-0 array that consists of 10 SSDs. The host runs on Linux OS 4.9.79. The FPGA FAC model is 200 MHz Xilinx VU9P. The FPGA provides 16 GB of memory and is connected to the host through PCIe Gen3 x16. A total of eight CUs are configured on the FPGA.

1) CU Acceleration Evaluation

We designed a unit test to compare the throughput during compaction when a single CU is used and when a single CPU is used. We loaded four paths with a total of 4 million records and triggered compaction. In different KV scenarios, the acceleration rate of the FPGA ranges from 200% to 500% during compaction.

Image for post
Image for post

2) System Performance Evaluation

To evaluate the performance, we use DbBench, with workloads distributed through zipf 1.0 and a default read-write ratio of 3:1. We load 32 subtables corresponding to an LSM tree, with each subtable containing 200 million data entries. We set the memory size to 70 GB, the key size to 8 bytes, and the value size to 32 bytes. This configuration is a common online configuration. The ramp-up period is 3,600 seconds and the evaluation period is 7,200 seconds for each test.

Image for post
Image for post
Image for post
Image for post

6) Summary

As application loads are constantly changing and the hardware environment is constantly evolving, storage engine development is seeking for the optimal balance between cost and performance.

Acknowledgments

This project was completed with support from the X-Engine team of Alibaba Database Product, the custom computing team of Alibaba Infrastructure Service (AIS), Damo Academy Database, Storage Lab, and Alibaba-Zhejiang University Joint Research Institute of Frontier Technologies (AZFT). I would like to express my gratitude to the researchers who participated in this work and the instructors who provided guidance on this paper. I believe that more cooperative achievements are forthcoming in the future.

Original Source:

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store