Self-built Kubernetes on Alibaba Cloud

By Anish Nath, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud’s incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.

Alibaba Cloud Container Service for Kubernetes is a fully-managed service compatible with Kubernetes to help users focus on their applications rather than managing container infrastructure. There are two ways to deploy Kubernetes on Alibaba Cloud, one through Container Service (built-in) and the other through an Elastic Compute Service (ECS) instance (self-built). If you are not sure which installation method suits your needs better, then refer to the documentation Alibaba Cloud Kubernetes vs. self-built Kubernetes.

For the most part, choosing Alibaba Cloud Container Service is the preferred choice as it saves time and reduces the complexity to maintain Kubernetes clusters. However, there may be cases where a manual installation is better suited to your needs. In this article, we will be setting up a self-built kubernetes on Alibaba Cloud Elastic Compute Service instances using Linux flavors (Centos7 and Ubuntu 16.04).


  1. A valid Alibaba Cloud account. If you don’t have one already, sign up to the Free Trial to enjoy $300 worth in Alibaba Cloud products.

Kubernetes Security Group Setup

These required ports needs to be opened in the pod security group, to do this:

  1. Log on to the ECS console.

Authorization ObjectProtocol TypePort RangeAuthorization Policy192.168.0.0/16TCP6443/6443Allow192.168.0.0/16TCP2379/2379Allow192.168.0.0/16TCP2380/2380Allow192.168.0.0/16TCP10250/10250Allow192.168.0.0/16TCP10251/10251Allow192.168.0.0/16TCP10252/10252Allow0.0.0.0/0TCP22/22Allow


  1. Alibaba Cloud Container Service and the pod CIDR block cannot overlap with the VPC CIDR block.

Connect to Your Alibaba Cloud Server

Locate the Internet IP address (Public IP address) associated with your Alibaba Cloud ECS Instance. There are other ways to connect to your ECS instance as well. Visit the official ECS documentation to learn more.

Before proceeding, stop and check whether:

  1. ECS public IP is enabled.

Setup Environment

This example utilizes two ECS instances running in the Alibaba Could environment and with these hostnames

Master Server nameMinion Clusterkube-masterkube-minion-1

Install Kubernetes-master and Kubernetes-minion

These are the minimum requirements to setup kubernetes ECS instance master and minion.

RequirementKubernetes-masterKubernetes-minionDisable system swap and SELinuxYYremove any swap entry from /etc/fstabYYnet.bridge.bridge-nf-call-iptables is set to 1YYInstall Docker & enable on restartYYInstall kubeadmYYInstall kubeletYNInstall kubectlYNConfigure docker cgroupsfsYNCreating Network Addons (flannel/Calico)YN

Disable system swap with swapoff -a. This will immediately disable swap and remove any swap entry from /etc/fstab

Disable SE Linux by setenforce 0

You should ensure net.bridge.bridge-nf-call-iptables is set to 1 in your sysctlconfig

cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
sysctl --system

If you are running CentOS or RHEL7, install Docker and enable on restart with these commands.

yum -y update
yum install -y docker

If you are running Ubuntu, install Docker and enable on restart with these commands.

apt-get update 
apt-get install -y

Start and enable Docker and check if docker service is running.

systemctl start docker
systemctl enable docker
systemctl status docker

Install Kubernetes

You will need to install these packages on all of your machines:

  1. kubeadm: the command to bootstrap the cluster.

Kubernetes Installation on CentOS7/RHEL7

Setting up Kubernetes yum repository

cat <<EOF > /etc/yum.repos.d/kubernetes.repo

Install Kubernetes in CentOS7/RHEL7

yum install -y kubelet kubeadm kubectl

Kubernetes Installation on Ubuntu

Setting up Kubernetes apt repository for Ubuntu

apt-get install -y apt-transport-https curl
apt-get install -y
curl -s | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb kubernetes-xenial main
apt-get update

Install Kubernetes in Ubuntu

apt-get install -y kubelet kubeadm kubectl

Running Kubernetes

Enable & start kublet

systemctl enable kubelet  
systemctl start kubelet

Verify that your Docker cgroup driver matches the kubelet config: (kube-master node)

docker info | grep -i cgroup
Cgroup Driver: cgroupfs

Configuration for cgroup drive is right in /etc/systemd/system/kubelet.service.d/10-kubeadm.conf


Reload daemon and restart kubelet: (kube-master node)

systemctl daemon-reload
systemctl restart kubelet

Flush Reset Kubernetes component (This will wipeout all the cluster config if exist any do not run this after the config create)

kubeadm reset -f

Set up Kubernetes Network add on

  1. For flannel to work correctly, — pod-network-cidr= has to be passed to kubeadm init.

Creating flannel Networks

Note apiserver-advertise-address is the IP of the kube-master

kubeadm init --service-cidr --kubernetes-version v1.11.0 --pod-network-cidr --apiserver-advertise-address

You should get information back on initiating commands as a normal user, as well as the network that you need to deploy as well as how to join worker nodes to the cluster.

To start using your cluster, you need to run the following as a regular user:  mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You can now join any number of machines by running the following on each node
as root:

Join the Cluster (kube-minnion)

kubeadm join --token 5m8qxr.46rpadiwt8fcka0v --discovery-token-ca-cert-hash sha256:b05a0b8849a57432247c06200864f5ce99d40ffdcae965293c0026204ef33da4

Run kubectl get nodes on the master to see this node join the cluster. It will few seconds to get your cluster in ready state

root@kube-master:kubectl get nodes
kube-master Ready master 2m v1.11.0
kube-minion Ready <none> 47s v1.11.0

Apply flannel Addons

kubectl apply -f

You will get output like this created created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.extensions/kube-flannel-ds created

Enable and restart kubelet engine

systemctl restart kubelet
systemctl status kubelet

Verifying the Installation

Verify the Cluster Information

root@kube-master: kubectl cluster-info 
Kubernetes master is running at
KubeDNS is running at

Verify the Services

root@kube-master:kubectl get services -n kube-system
kube-dns ClusterIP <none> 53/UDP,53/TCP 2m

Create Tokens

[root@kube-master ~]kubeadm token create 
I0710 04:08:37.149017 8685 feature_gate.go:230] feature gates: &{map[]}

That’s it! You have successfully configured a self-built Kubernetes cluster on Alibaba Cloud. To learn more about Kubernetes on Alibaba Cloud, visit Alibaba Cloud Container Service for Kubernetes.Reference:

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.