Implementing OSS Multipart Download for Mobile Devices

11.11 The Biggest Deals of the Year. 40% OFF on selected cloud servers with a free 100 GB data transfer! Click here to learn more.

When a user watches a video using a video app on his/her mobile phone, the video app pre-fetches the content for a seamless user experience. Normally, the app will switch the network from mobile data to Wi-Fi during the download to help users reduce data consumption. When the download resumes, unfinished parts can be downloaded separately from the already downloaded parts, which saves time and traffic.

In this article, we will discuss the implementation of this function on mobile terminals using the Alibaba Cloud Object Storage Service (OSS) multipart download feature.

Technical Points

Support for Range headers is added in HTTP 1.1 to specify the scope for getting data. The formats of Range headers generally include:

  1. Range: bytes=100- Start from the 101st byte until finished.
  2. Range: Bytes=100-200 Specify the length from the beginning to the end, remember that Range is counted from 0, so this requires the server to start from the 101st byte to the 201st byte. This is generally used for sharding a particularly large file, such as video.
  3. Range: Bytes=-100 If the range does not specify a starting position, it requires the server to transfer the last 100 bytes of content, rather than 100 bytes starting from byte 0.
  4. Range: bytes=0-100, 200-300 Multiple ranges of content can be specified at the same time, which is not common.

In addition, when the multipart is resumed, it is necessary to verify whether the file on the server has changed. At this time, the If-Match header is used. If-Match corresponds to the value of Etag.

When the client initiates the request with Range and If-Match attached in the header, the OSS server will verify the Etag value in 'If-Match' after receiving the request. If it does not match, it will return a 412 precondition status code.

The OSS server supports the Range, If-Match, If-None-Match, If-Modified-Since, If-Unmodified-Since for the open API getObject, so we can implement the multipart download function of OSS resources on the mobile terminal.

Best Practices

First let’s look at the flow chart:

Image for post
Image for post

Our goal is to implement “pausable” or “resumable” downloads by using OSS multipart download feature.

Image for post
Image for post

Here is an example of iOS download implementation. The reference code is as follows. It is for reference only, and cannot be used for production.

Shown above is the processing logic for receiving data from the network, with DownloadService at the core of the download logic. In the URLSession:dataTask:didReceiveData, the received network data is appended to the file, and the download progress is updated. In the URLSession:task:didCompleteWithError, determine whether the download task is completed, and then return the result to the upper layer service. In the URLSession:dataTask:didReceiveResponse:completionHandler proxy method, the object-related information is used, for example etag for the multipart resume precheck, and content-length for calculating the download progress.

The above section is the definition of DownloadRequest.

This section is the call in an upper layer service. The checkpoint can be obtained from the failure callback for both pausing and canceling the upload. The checkpoint can be passed to the DownloadRequest when resuming the download, which is used by the DownloadService to perform a consistency check.

For implementing the multipart download of OSS Object in Android, see the open source project The Multipart Download Function Implemented Based On Okhttp3. The following shows how to use this sample project to download OSS resources.

Improvements

For If-Range in HTTP 1.1, let’s look at the description of the header:

The If-Range HTTP request header makes a range request conditional: if the condition is fulfilled, the range request will be issued and the server sends back a 206 Partial Content answer with the appropriate body. If the condition is not fulfilled, the full resource is sent back, with a 200 OK status.

This header can be used either with a Last-Modified validator, or with an ETag, but not with both.

The most common use case is to resume a download, to guarantee that the stored resource has not been modified since the last fragment has been received.

Advantages of using If-Range: The client only needs one network request. When the condition is determined as failure, the If-Unmodified-Since or If-Match mentioned above will return 412 pre-condition check failure status code, and the client has to make another request to get the resource.

OSS Server currently does not support the field If-Range. When using NSURLSessionDownloadTask in iOS for multipart download, If-Range will be sent to the server to verify whether the file has been modified. So currently NSURLSessionDownloadTask cannot be used for multipart download.

Reference: https://www.alibabacloud.com/blog/implementing-oss-multipart-download-for-mobile-devices_594123?spm=a2c41.12228633.0.0

Written by

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store