Managing ECS Instances with Ansible Dynamic Inventory

By Anish Nath, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud’s incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.

If your Alibaba Cloud Elastic Compute Service (ECS) inventory fluctuates over time, with hosts spinning up and shutting down in response to business demands, the static inventory solutions will not serve your needs. You may need to track hosts from multiple sources like CMDB, manual query and compilation and all other tedious tasks.

Ansible integrates all of these options via a dynamic external inventory system using Ansible Alicloud Module inventory scripts which can be found here https://raw.githubusercontent.com/alibaba/ansible-provider/master/contrib/inventory/alicloud.py

In this article we will learn how to:

  • Configure ECS Dynamic Scripts
  • Query ECS hosts
  • Integrate ECS with Ansible dynamic inventory
  • Use instance filters
  • Achieve security compliance using ECS dynamic scripts

Prerequisites

Log in to your Alibaba Cloud ECS controller node or your managed node and install the following components:

$ ssh root@47.254.17.249 root@47.254.17.249's password:
Welcome to Alibaba Cloud Elastic Compute Service !
$ sudo pip install ansible
$ sudo pip install ansible_alicloud

Note: Make sure your NTP is in sync, otherwise you will be seeing API call failure

In case you are new to Alibaba Cloud, you can get $10 worth in credit through my referral link to get started on an ECS instance.

Authentication

To pass authentication credentials as environment variables:

export ALICLOUD_ACCESS_KEY='XXXXXXXXXXXXX' 
export ALICLOUD_SECRET_KEY='YYYYYYYYYYYYYZZZZZZZZZZZZ'

The RAM key associated with this account should be having appropriate permission to query Alibaba Cloud resources. The granularity to be determined by the system security professional at minimum it should be having AliyunECSReadOnlyAccess

Inventory Script

$wget https://raw.githubusercontent.com/alibaba/ansible-provider/master/contrib/inventory/alicloud.py 
$chmod +x alicloud.py
$sudo cp alicloud.py /etc/ansible/hosts

Download the sample Ansible Alibaba Cloud ECS dynamic inventory, modify it to suit your needs and copy it to /etc/ansible/alicloud.ini:

$ wget https://raw.githubusercontent.com/alibaba/ansible-provider/master/contrib/inventory/alicloud.ini  
$vi alicloud.ini
$sudo cp alicloud.ini /etc/ansible/

You can test the Alibaba Cloud dynamic inventory script manually to confirm it is working as expected:

$ ./alicloud.py --list

After a few moments you should see some JSON output with information about your ECS instances.

[Instance:i-rj9djh1xn7tava07qtam]
{
"_meta": {
"hostvars": {
"i_rj9djh1xn7tava07qtam": {
"ansible_ssh_host": "47.254.17.249",
"auto_release_time": "",
"availability_zone": "us-west-1b",
"block_device_mapping": [
{
"attach_time": "2019-01-04T04:43:26Z",
"delete_on_termination": true,
"device_name": "/dev/xvda",
"status": "in_use",
"volume_id": "d-rj9euf5u1ph9l348lre8"
}
],
"cpu": 1,
"creation_time": "2019-01-04T04:43Z",
"credit_specification": "Standard",
"deletion_protection": false,
"deployment_set_id": "",
"description": "demo",
"eip": {
"allocation_id": "",
"internet_charge_type": "",
"ip_address": ""
},
"expired_time": "2099-12-31T15:59Z",
"gpu": {
"amount": 0,
"spec": "",
"specification": ""
},
"host_name": "demo",
"id": "i-rj9djh1xn7tava07qtam",
"image_id": "alinux_17_01_64_20G_cloudinit_20171222.vhd",
"instance_charge_type": "PostPaid",
"instance_id": "i-rj9djh1xn7tava07qtam",
"instance_name": "demo",
"instance_type": "ecs.t5-lc2m1.nano",
"internet_charge_type": "PayByTraffic",
"internet_max_bandwidth_in": 100,
"internet_max_bandwidth_out": 5,
"io_optimized": true,
"memory": 512,
"network_interfaces": [
{
"mac_address": "00:16:3e:00:46:ea",
"network_interface_id": "eni-rj99fi72su03d5iqpv26",
"primary_ip_address": "172.20.240.111"
}
],
"osname": "Aliyun Linux 17.1 64\u4f4d",
"ostype": "linux",
"private_ip_address": "172.20.240.111",
"public_ip_address": "47.254.17.249",
"resource_group_id": "",
"state": "running",
"tags": {
"tool": "ansible"
},
"user_data": "",
"vpc_id": "vpc-rj9mhs1pv5tbc31khpssv",
"vswitch_id": "vsw-rj9bt4ec9zqauhr8vig9q"
}
}
},
"alicloud": {
"children": [
"i_rj9djh1xn7tava07qtam"
]
},
"alinux_17_01_64_20G_cloudinit_20171222.vhd": [
"i_rj9djh1xn7tava07qtam"
],
"i-rj9djh1xn7tava07qtam": [
"i_rj9djh1xn7tava07qtam"
],
"i_rj9djh1xn7tava07qtam": [
"47.254.17.249"
],
"security_group_security_group_id": [
"i_rj9djh1xn7tava07qtam"
],
"subnet_vsw_rj9bt4ec9zqauhr8vig9q": [
"i_rj9djh1xn7tava07qtam"
],
"tag_tool_ansible": [
"i_rj9djh1xn7tava07qtam"
],
"type_ecs_t5_lc2m1_nano": [
"i_rj9djh1xn7tava07qtam"
],
"us-west-1": [
"i_rj9djh1xn7tava07qtam"
],
"us-west-1b": [
"i_rj9djh1xn7tava07qtam"
],
"vpc_id_vpc_rj9mhs1pv5tbc31khpssv": [
"i_rj9djh1xn7tava07qtam"
]
}

Refreshing the Cache

$ /alicloud.py --refresh-cache

To see the complete list of variables available for an instance, run the script by itself:

Once you confirm the dynamic inventory script is working as expected, you can tell Ansible to use the alicloud.py script as an inventory file, as illustrated below:

[root@demo ~]# ansible -i alicloud.py all -m ping -k -u root 
SSH password:
[WARNING]: Found both group and host with same name: i_rj9djh1xn7tava07qtam
i_rj9djh1xn7tava07qtam | UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh: ssh: Could not resolve hostname i_rj9djh1xn7tava07qtam: Name or service not known\r\n",
"unreachable": true
}
47.254.17.249 | SUCCESS => {
"changed": false,
"ping": "pong"
}

ECS Dynamic Inventory Script Configuration

  • Regions: A group of all instances in an ECS region for example cn-beijing,eu-central-1,ap-southeast-1,us-east-1
  • Tags: Each instance can have a variety of key/value pairs associated with it called Tags. Each key/value pair is its own group of instances, again with special characters converted to underscores, in the format tag_TAGNAME
  • HOST VAR: How the VM to be queried

Edit the file alicloud.ini and change the hostname_variable

from
hostname_variable = instance_id
to
hostname_variable = instance_name

Run the script again

[root@demo ~]# ansible -i alicloud.py all -m ping -k -u root 
SSH password:
[WARNING]: Found both group and host with same name: demo demo | SUCCESS => {
"changed": false,
"ping": "pong"
}
47.254.17.249 | SUCCESS => {
"changed": false,
"ping": "pong"
}

What about the private VM’s which doesn’t have the public address, how to manage those VM’s. this can be achieved through modifying the alicloud.ini file and changing the destination_variable to use private_ip_address

destination_variable=private_ip_address

The below ansible query will use ECS private address to manage the VM

[root@demo ~]# ansible -i alicloud.py all -m ping -k -u root 
SSH password:
[WARNING]: Found both group and host with same name: demo
demo | SUCCESS => {
"changed": false,
"ping": "pong"
}
172.20.240.111 | SUCCESS => {
"changed": false,
"ping": "pong"
}

instance_states

There are five ECS Instances states associated to the ECS VM’s [‘pending’, ‘running’, ‘starting’, ‘stopping’, ‘stopped’]. By default, only ECS instances in the running status are returned, in order to control these option modify the alicloud.ini file and tune these setting according to your need.

all_instances = True
# instance_states = pending, running, starting, stopping, stopped

Security Compliance

*/30 * * * *  ansible-playbook ssh.yml -i alicloud.py

and refresh the cache every four hour through cron job to have an updated List of Inventory, Tune it according to your need.

**0 0 */4 ? * ***   alicloud.py --refresh-cache

Conclusion

  • Alibaba Cloud ECS Dynamic Scripts will have always right IP address , if changed manually (EIP assigned to different VM)
  • Never supply Alibaba Cloud credentials in alicloud.ini is not recommended , It is strongly recommended using environment variable

Reference:https://www.alibabacloud.com/blog/managing-ecs-instances-with-ansible-dynamic-inventory_594718?spm=a2c41.12820850.0.0

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.