How Fast Is Alibaba Cloud CDN: An In-Depth Performance Analysis

By Valerie YJ Lee, Solutions Architect

The purpose of this study is to evaluate the performance factors of Alibaba Cloud’s CDN service by region by measuring object and web page load time. We will be running two types of tests, as described below.

Object Load Test

This test is that measures the NETWORK PERFORMANCE by periodically loading certain objects from distributed CDN nodes.

Test URL examplehttp://www.abc.com/img/product/test.png (QT011_100K)Test toolCatchPointTest nodesKR (Seoul), HK (Hong Kong), TW (Taipei), JP (Tokyo), SG (Singapore), CN (Beijing, Shanghai), VN (Hanoi), ID (Jakarta), MY(Kuala Lumpur), TH (Bangkok), US (New York, San Jose), AU (Sydney), CA (Toronto) — 15

The test was performed twice for comparing the improvements before and after CDN tuning.

Page Load Test

This test is similar with object load test, but this measures the whole page response time in various locations. Therefore, this evaluates the UX LEVEL PERFORMANCE rather than network environment.

Test URL example 1http://www.abc.com/img/Prd.mall?spm=a2c41.11461447.0.0.3fea52e8Lmsfng&method=getPDtail&prdNo=1(QT021)
http://www.abc.com/img/Search.mall?spm=a2c41.11461447.0.0.3fea52e8Lmsfng&method=getSDtail&isGnb=N(QT022)Test toolCatchPointTest nodesKR (Seoul), HK (HongKong), TW (Taipei), JP (Tokyo), SG (Singapore), CN (Beijing, Shanghai), VN (Hanoi), ID (Jakarta), MY(Kuala Lumpur), TH (Bangkok), US (New York, San Jose), AU (Sydney), CA (Toronto), Dubai, GM (Frankfurt), PH (Manila) — 18Test URL example 2 http://www.xyz.com/index.html?spm=a2c41.11461447.0.0.3fea52e8LmsfngTest toolGomezTest nodesHK (HongKong), JP (Tokyo), CN (Beijing, Shanghai), TW (Taipei)

Object Load Test

Before reading, make sure that you remember these factors that indicate good performance

  • A smaller gap between median(green line) and the 75 percentile (khaki line)
  • Smaller occurrences below median (converges to zero)
  • Less frequent occurrences of over 75 percentile

Before Tuning

Image for post
  1. Median-75 percentile distance: around 700
  2. 0–75 percentile: around 50%
  3. The proportion of 75 percentile or more: around 25%

After Tuning

Image for post
  1. Median-75 percentile distance: around 600
  2. 0–75 percentile: around 40%
  3. The proportion of 75 percentile or more: around 17%

A/B test comparison

  1. Median-75 percentile distance decreased from around 700 to around 600. This shows that “Local deviation” slightly decreased.
  2. 0–75 percentile increased from 50% to around 40%. This means that “good performance area” increased.
  3. The proportion of 75 percentile or more decreased from 25% to 17%. This describes that reduction of “bad performance area”

Summary of Results

  1. Result table by testing nodes
  • A means “Before tuning” and B means “After tuning”
  • Response = DNS + Connect + Wait + Load
Image for post

1) Response time in HK, JP, TW and SG are very good. In US San Jose, Shanghai and Bangkok, performance is good (Shanghai has no CDN)
2) Beijing supplement needed — After ICP completion
3) ID and MY need to be supplemented

Page Load Test

Test by CatchPoint

POC vs. Production

The below table shows performance in POC environment (without CDN vs. Alibaba CDN vs. third-party vendor CDN)

Image for post

These charts and table show performance in production environment.

Image for post

In production environment, the response time decreased from 26 seconds to 12 seconds (Test URL 1) and from 12 seconds to 10 seconds with Alibaba CDN.

Detailed performance data in production environment

Image for post

The red dots mean “bytes limit exceeds” error of own test tool.

  1. Both of two tests’ Median-75 percentile distance is less than 2.5 seconds compared to total distribution (31 seconds). This converts to overall quality average.
  2. Each test’s 0–75 percentile is analyzed to be around 75% and 60%. Comparing two tests, QT022 provides better UX.
  3. Both of two tests are analyzed as a percentage of more than 75 percentile, but QT022 is better
Image for post

Summary of Results

  1. Generally, the VPN is quite good quality in most areas (including Europe and the Middle East)
  2. Beijing, Shanghai need to complement — After ICP is completed
  3. Need to supplement MY — Planed to significantly reinforce the CDN nodes

Test by Gomez

Comparison of Alibaba Cloud CDN with Third-Party Vendor

Image for post
Image for post

Summary of Results

  1. Alibaba CDN shows superior performance compared to other CDNs for all indicators in China
  2. Alibaba CDN maintains performance similar to other companies in mainland China and other Asian countries (Taiwan, Hong Kong, Japan)
Image for post

Conclusion

Alibaba Cloud CDN provides excellent network performance and UX quality, and has a local and temporal distribution of response time. In particular, it can be seen that the response time in China is greatly improved with our CDN product. We are constantly increasing the number of nodes in various countries around the world, so our customers can expect better response times globally.

Reference:https://www.alibabacloud.com/blog/how-fast-is-alibaba-cloud-cdn-an-in-depth-performance-analysis_594652?spm=a2c41.12741348.0.0

Written by

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store