Uploading and Downloading Complex MaxCompute Data Using Tunnel SDK

By Yunhua

This article shows you how you can upload complex data to MaxCompute through Tunnel SDK. First, we introduce the types of complex MaxCompute data:

Complex Data

MaxCompute uses the ODPS 2.0-based SQL engine to enrich the support for complex data. MaxCompute supports the ARRAY, MAP, and STRUCT types, which can be used in a nested manner, and provides built-in functions.

Image for post
Image for post

Construction and Operation Functions for Complex Types

Image for post
Image for post

Introduction to Tunnel SDK

MaxCompute Tunnel is the data channel of MaxCompute. You can use it to upload data to or download data from MaxCompute.

TableTunnel is the entry class to access MaxCompute Tunnel. It only supports upload and download of table data (non-view).

Session is a process for uploading and downloading a table or a partition. A session is composed of one or more HTTP requests to Tunnel RESTful APIs.

The session ID is used to identify a session. The timeout time of a session is 24 hours. If massive data is transferred for more than 24 hours, you need to split it into multiple sessions.

Sessions TableTunnel.UploadSession and TableTunnel.DownloadSession are responsible for data upload and download.

TableTunnel provides methods to create UploadSession DownloadSession objects.

  • A typical table data upload process:

1) Create TableTunnel

2) Create UploadSession

3) Create RecordWriter and write the record

4) Submit the upload operation

  • A typical table data download process:

1) Create TableTunnel

2) Create DownloadSession

3) Create RecordReader and read the record

Construct Complex Data Types Based on Tunnel SDK

Sample code:

Download Complex Data Types from MaxCompute

Sample code:

Running Example

The complete code is as follows:

Reference:https://www.alibabacloud.com/blog/uploading-and-downloading-complex-maxcompute-data-using-tunnel-sdk_594739?spm=a2c41.12825644.0.0

Written by

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store