Building An Immutable Cloud Infrastructure On Alibaba Cloud


In a traditional cluster environment, engineers and administrators are randomly sshing into their servers, installing packages or even tweaking configuration on a server-by-server basis. Applications are deployed manually. Dependencies are introduced accidentally. Configurations are modified and drifted unpredictably. Evidently human factors will sooner or later become a catastrophe.

Immutable Application Platform

Application Platform here can refer to specific runtimes like Java, Python, etc.., or generic ones like container technologies. In this section we will only talk about Docker considering its popularity and relevancy.

Immutable File System

While Docker and other container technologies provides application runtime immutability, which cannot cover all the scenario, as Docker, at the file system layer, uses OverlayFS or UnionFS driver which are not persistent and only suit for stateless and volatile applications. Although this is regarded as one of the best practices for micro-services based architecture, we cannot assume it will be best for all applications, typically the legacy ones. In this section we will introduce OSTree, which works in Linux userspace as an abstraction tier and can be on top of any existing file systems, and basically without any dependency.

env = dev
port = 8000
/var/demo/!/usr/bin/env python import SimpleHTTPServer
import SocketServer
import ConfigParser
config = ConfigParser.RawConfigParser()'/etc/demo/demo.conf')
env = config.get('main', 'env')
port = config.getint('main', 'port')
Handler = SimpleHTTPServer.SimpleHTTPRequestHandler httpd = SocketServer.TCPServer(("", port), Handler) print("serving %s at port %d" % (env, port))
$ yum install -y ostree
$ mkdir -p /etc/demo && cd /etc/demo
$ ostree --repo=.demo init
$ ostree --repo=.demo commit --branch=dev ./
$ ostree --repo=.demo refs
$ ostree --repo=.demo ls dev
d00755 0 0 0 /
-00644 0 0 29 /demo.conf
d00755 0 0 0 /.rdemo
$ ostree --repo=.demo cat dev /demo.conf
env = dev
port = 8000
chmod +x /var/demo/
serving dev at port 8000
$ cat /etc/demo/demo.conf
env = uat
port = 80
$ ostree --repo=.demo commit --branch=uat ./
serving uat at port 80
$ ostree --repo=.demo cat dev /demo.conf
env = dev
port = 8000
$ ostree --repo=.demo cat uat /demo.conf
env = uat
port = 80

Immutable Package Management

OSTree enables atomic upgrades of complete file system trees, and its subsystem OSTree-RPM primarily provides package management functionalities at a higher level. If you are a system admin, operating system patching is a struggle. Taking the example of OpenSSL, without the latest patch, your infrastructure may be exposed to security volunerlity like the heartbleed bug. With the latest patch, you might encounter a large range of compatability issues. With rpm-ostree, you may feel confident to upgrade and test, because reverting the package system is so painless here.

$ rpm-ostree status
$ rpm-ostree upgrade
$ systemctl reboot
$ openssl version
OpenSSL 1.0.1e-fips 11 Feb 2013
$ rpm-ostree install openssl
Checking out tree 67d659b... done
Inactive requests:
openssl (already provided by openssl-1:1.0.2k-12.el7.x86_64)
Enabled rpm-md repositories: base updates extras
Updating metadata for 'base': [=============] 100%
rpm-md repo 'base'; generated: 2018-11-25 16:00:34
Updating metadata for 'updates': [=============] 100%
rpm-md repo 'updates'; generated: 2019-01-24 13:56:44
Updating metadata for 'extras': [=============] 100%
rpm-md repo 'extras'; generated: 2018-12-10 16:00:03
Importing metadata [=============] 100%
Resolving dependencies... done
Checking out packages (2/2) [=============] 100%
Running pre scripts... 0 done
Running post scripts... 1 done
Writing rpmdb... done
Writing OSTree commit... done
Copying /etc changes: 22 modified, 8 removed, 41 added
Transaction complete; bootconfig swap: no; deployment count change: 0
Freed: 39.0 MB (pkgcache branches: 2)
Run "systemctl reboot" to start a reboot
$ openssl version
OpenSSL 1.0.2k-fips 26 Jan 2017
$ rpm-ostree rollback
Moving '67d659bc257b7d47f638f9d7d2146401b85eec7c7eef0122196d72c70553ae66.0' to be first deployment
Transaction complete; bootconfig swap: no; deployment count change: 0
Run "systemctl reboot" to start a reboot
$ systemctl reboot $ openssl version
OpenSSL 1.0.1e-fips 11 Feb 2013

Immutable Provisioner

Generally, a provisioner may refer to a host in a hypervisor pool, a compute node in OpenStack, or a Worker Node in Kubernetes. These are like the infrastructures under the infrastructures, the host of the host, or the machine who builds machines.

$ atomic install
Pulling ...

Creating directory at /host//etc/pki/rsyslog
Installing file at /host//etc/rsyslog.conf
Installing file at /host//etc/sysconfig/rsyslog
Installing file at /host//etc/logrotate.d/syslog
$ atomic run
docker run -d --privileged --name rsyslog --net=host --pid=host -v
/etc/pki/rsyslog:/etc/pki/rsyslog ...


In this article, we briefly introduced how Docker, ostree, ostree-rpm and Atomic Host can help you build immutable infrastructure layers, from application platform and file system to package management and provisioner, on Alibaba Cloud, in order to lower the complexity of system and risk of human factors by making everything consistent.

Original Source



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alibaba Cloud

Alibaba Cloud

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website: