DRDS Read-only Instance for Complex SQL Queries

Main Features of DRDS

Self-Developed MPP Multi-Machine Concurrent Computing Engine ¨CFireworks

The DRDS read-only instance hosts an SQL execution engine (Fireworks) that features complete multi-machine concurrent processing capabilities. The engine is significantly different from the SQL execution engine hosted on the DRDS primary instance.

An Optimizer Customized for the Multi-Machine Concurrent Execution Mode

The original optimizer for the DRDS primary instance focuses on OLTP scenarios. Its core concept is to push all computing tasks to its physical sub-databases for execution if possible. Its purpose is to fully utilize the computing resources of physical databases and avoid massive data flow for quick response.

  1. Execute as many complex computing tasks (such as join, aggregate, and sort tasks) as possible on the execution engine of the DRDS itself, realizing computing speedup and scalability through the Fireworks computing cluster.
  2. Push lightweight computing tasks (such as project and filter tasks) to the physical database to reduce data pulling costs.

Direct Analysis based on Online Data

Emerging Internet services, represented by the new retail business, are springing up. The industry features not only real-time OLTP requirements, but also complex quasi-real time OLAP requirements, to support real-time decision making.

SQL Compatibility with Clear Boundaries

The DRDS read-only instance is comprehensively compatible with the SQL query syntax of the DRDS primary instance and is highly consistent with DRDS V 5.3 in terms of SQL compatibility and SQL supporting boundaries.

Flexible Product Experience

The DRDS read-only instance automatically synchronizes the account privilege information of the DRDS primary instance and supports the native VPC. You can enable it on both the Intranet or Internet and adjust the configuration flexible based on the business situation. It will foster linear improvement on data processing capabilities.

Technical Architecture Overview

The overall architecture of the DRDS read-only instance is basically the same as that of the DRDS primary instance. Only the query layer is changed, being added with the MPP execution engine and the corresponding optimizer, as shown below:

Application Scenarios

In general, the DRDS read-only instance is applicable to complex queries in high data volume levels featuring low concurrency and high latency, such as data analysis and report scenarios, typical for its large number of join, aggregate, and sort operations performed on a large volume of data.

  1. Some complex SQL queries involving join, aggregate, and sort operations may require secondary computing in the DRDS execution engine because they cannot be completely pushed to the physical databases, and yet such computing cannot be executed due to the memory limitation of the stand-alone execution engine.
  2. Complex computing parts of an SQL query can be pushed to physical databases. However, if the query involves a large volume of data, due to the high pressure on physical databases, the OLTP service is affected, or the response time is too slow to meet the requirements.

Summary

For a long time, the DRDS has been unable to provide good support for complex queries involving large data scales or implement linear scaling upon its local computing capabilities through physical resource expansion due to the limitation of the execution engine in stand-alone architecture.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alibaba Cloud

Alibaba Cloud

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website:https://www.alibabacloud.com