Alibaba Cloud DevOps Cookbook Part 1 — CLI, SDK, SSH, SFTP

By John Hanley, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud’s incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.

This section is the beginning of applying DevOps to Alibaba Cloud. I plan to start simple and build simple tools and then following up with more complex real-world DevOps tools. The key is to start with the basics and understand the low-level features.

Now that I have a test website that is load balanced and has auto scaling, I would like to learn more about the Alibaba Cloud CLI and Python SDK. During development I often need to make changes to files that I publish on my ECS instances. Since the auto scaling group is built from an image, changing the image takes effort and time. During testing, I want to do rapid-fire edit / deploy / debug / improve. This means that I need a quick way to upload files to my ECS instances all at once.

Part 1. Working with the CLI

Using an SSH GUI based program to copy files is easy. However, with auto scaling, I don’t know how many instances are running nor their IP addresses. These variables can change at any time. Today, I will start with just the CLI and then improve by combining the CLI and Python to create a program that will do the following:

  1. Get a list of ECS instances attached my load balancer (SLB).
  2. For each ECS instance, get the public IP address.
  3. For each ECS instance upload a file using SFTP.

This sounds simple, but this will take some work.

To work with the SLB, we will need to install the SDK for SLB. Execute the following command while c:Python27Scripts is in your path:

# pip install aliyun-python-sdk-slb

Let’s list the Server Load Balancers that are in our account. Replace the region with your region.

# aliyuncli slb DescribeLoadBalancers

The result looks like this:

We are interested in the Load Balancer ID:

"LoadBalancerId": "lb-abcdef1a7bf644x6s1fzg",

Next, I want to display the ECS instances that are running under the SLB.

# aliyuncli slb DescribeLoadBalancersRelatedEcs --LoadBalancerId lb-abcdef1a7bf644x6s1fzg

The result looks like this:

We are interested in the VmName for each instance:

Next, I want to display the IP Address for the first ECS instance. Note: this command is one line.

# aliyuncli ecs DescribeInstanceAttribute --InstanceId i-0123451kxyhfs89rykzg --filter PublicIpAddress.IpAddress[0]

The result looks like this:

"47.254.99.120"

Next, I want to display the IP Address for the other ECS instance. Note: this command is one line.

# aliyuncli ecs DescribeInstanceAttribute --InstanceId i-01234538slefk99bps9h --filter PublicIpAddress.IpAddress[0]

The result looks like this:

"47.254.99.121"

That was easy. Now all that is required is to use SSH and upload my files to the ECS instances using the SFTP protocol. Since I use the Bitvise SSH program, I use the Bitvise SFTP client to upload files. The following command is for Windows. You will need your Key Pair that you specified when created the auto scaling group.

Again, that was easy. However, tomorrow or next month, the instances running may be different and they might have different IP addresses. This means that I will have to edit my batch script that figures out the ECS instance IDs and Public IP addresses. An improved solution is to combine the CLI with Python to automate this.

Part 2. Automate the CLI with Python

You can download a zip package of the files mention in this part. Download slb_upload.zip

The python code is Python 3.x. The python version is 3.6.1.

The file slb_upload.py is the main program. This program takes either 3 or 4 arguments. If you only have one Server Load Balancer, then you do not need to specify the fourth parameter. If you do have more than one SLB, then this is the SLB ID.

Usage: slb_upload local_path remote_path [slb_id]

Example command:

python slb_upload c:\work\websites\neoprime.xyz\hello.html /var/www/html/hello.html

The file myslb.py contains routines to interface with SLB and ECS. This file contains the following functions:

  1. ECS_UploadFIle — Uploads a file using SFTP to an ECS instance.
  2. ECS_GetPublicIp — Takes a VmName from SLB and gets the Public IP Address for the ECS instance.
  3. SLB_GetServerLoadBalancers — Returns an array of Sever Load Balancers
  4. SLB_GetBackendInstances — Returns an array of ECS instances attached to an SLB

Source file: myslb.py

Source file: slb_upload.py

Part 3. Automating with Python and the SDKs

You can download a zip package of the files mention in this part. Download slb_upload_python.zip

The code in Part 2 demonstrates how rapidly you can use Python and the CLI to automate tasks. To start to really understand the Alibaba Cloud Tools and SDKs, the next logical step is to convert the program into 100% SDK based instead of using the CLI. This will happen next.

When using the Python SDK, the first step is to setup your credentials. The Alibaba CLI creates the directory .aliyuncli in your HOME directory. In this directory are two files:

  1. configure — This file contains items like region, preferred output such as json, etc.
  2. credentials — This file contains your Access Key and Access Key Secret

At the CMD Prompt execute this command:

aliyuncli configure

The CLI will prompt you for several items:

  1. Aliyun Access Key ID
  2. Aliyun Access Key Secret
  3. Default Region ID
  4. Default output format

For the first two items, these are the keys that you saved when you create the new user. For the region, either leave it blank or specify your preferred region. I used us-west-1. For the output format, I prefer json.

Next we will port the myslb.py module to replace the CLI command line calls with Python SDK calls.

Source file: myslb.py

The file mycred_acs.py contains very simple routines to load the credentials from the configure and credentials files mentioned above. I am a firm believer in never hard-coding your credentials in source code.

Source file: mycred_acs.py

And now for the main program slb_upload.py. This was modified a bit to support passing credentials to the ECS and SLB support functions.

Source file: slb_upload.py

Continue reading Alibaba Cloud DevOps Cookbook Part 2

Developer Documents

Command Line Interface (CLI):

  1. Alibaba Cloud CLI

Server Load Balancer (SLB):

  1. Server Load Balancer Document Page
  2. Server Load Balancer Developer Guide

Elastic Compute Service (ECS):

  1. Elastic Compute Service Document Page
  2. Elastic Compute Service Developer Guide

Reference:

https://www.alibabacloud.com/blog/alibaba-cloud-devops-cookbook-part-1-%E2%80%93-cli%2C-sdk%2C-ssh%2C-sftp_593714?spm=a2c41.11623130.0.0

Written by

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store