Use a Local Disk Through LocalVolume Provisioner in a Kubernetes Cluster
By Kan Junbao
Introduction
Alibaba Cloud provides local disk configurations for some Elastic Compute Service ECS instances. Local disks have the advantages of low latency, high random IOPS, high throughput, and high cost effectiveness, and has great advantages in some applications with high performance requirements.
In the Kubernetes system, local disks can be used through HostPath, LocalVolume, and other types of PV.
- HostPath: The volume itself does not contain scheduling information. If you want to fix each pod on a node, you need to configure scheduling information, such as nodeSelector, for the pod;
- LocalVolume: The volume itself contains scheduling information, and the pods using this volume will be fixed on a specific node, which can ensure data continuity.
This document describes how to use the Alibaba Cloud local disk using LocalVolume.
- Use LocalVolume Provisioner to automatically create LocalVolume, and the PV comes with the information about the node to which it belongs.
- Create a PVC to consume LocalVolume. Pods using this PVC will be scheduled to the node specified by the PV.
How Provisioner Works
1) Introduction to Local Disks:
To learn more about ECS local disks, refer to Local disks
Local disks cannot be mounted or unmounted;
Local disks and ECS must coexist. The number of local disks is determined when ECS is created;
The type of the machine determines the number of local disks, which can be obtained through API;
By default, local disks are mounted from /dev/vdb, which may be different from custom images.
2) Mounting Principle:
Provisioner mounts local disks to the target directory at startup:
The number of local disks in this machine can be queried through API;
Local disks are mounted from the initiating device (/dev/vdb, by default) to the target directory in turn (the initiating device can be configured);
The device detects the file system. If the device does not format the file system, a file system is created;
Provisioner polls and checks the target directory in real time, and creates a PV based on the target directory list:
When a PV is created, StorageClass can support the configuration of reclaimPolicy and volumeBindingMode;
After LocalVolume PV is deleted, it is automatically created again;
PV name: local-pv-{node name}-{device Name}. The PV name will be added to the label;
Deploy Alibaba LocalVolume Provisioner
1) Create ConfigMap:
The configuration parameters of Provisioner are placed in the ConfigMap, provisioner-config:
vendor: optional. If it is alibabacloud, the Alibaba Cloud local disk is enabled;
hostDir: the mount directory of the local disk on the host;
mountDir: the mount directory of the local disk on the container after it is mounted to the host;
volumeMode: the mount mode, supporting file systems and blocks;
fsType: the file system type, which defaults to ext4;
deviceStartWith: optional. It defaults to vdb, the starting name of the local disk;
mkFSOptions: optional. It is null by default, indicating that the parameter after “mkfs. *” is performed;
mountOptions: optional. It is null by default, indicating that the parameter of “mount -o” is performed;
apiVersion: v1
kind: ConfigMap
metadata:
name: local-provisioner-config
namespace: kube-system
data:
storageClassMap: |
local-volume:
vendor: alibabacloud
hostDir: /mnt/disks
mountDir: /mnt/disks
blockCleanerCommand:
- "/scripts/shred.sh"
- "2"
volumeMode: Filesystem
fsType: ext4
deviceStartWith: vdb
mkFSOptions: ""
mountOptions: "nodelalloc"2) Create StorageClass
LocalVolume StorageClass does not actually go to Provisioner PV, but is used to define configuration details, such as reclaimPolicy and volumeBindingMode.
reclaimPolicy: It supports Retain and Delete;
volumeBindingMode: It supports Immediate and WaitForFirstConsumer.
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: local-volume
provisioner: kubernetes.io/no-provisioner
reclaimPolicy: Retain
volumeBindingMode: WaitForFirstConsumer3) Create a Provisioner Service
For deployment through Daemonset, set hostPID and hostNetwork to true;
The privileged permission is added;
Deployment through AK or STS is supported;
The AK-free STS function is supported;
The node needs to configure the RAM role information, and the role has access to the cloud disk.
When using the STS function, you do not need to configure ACCESS_KEY_ID and ACCESS_KEY_SECRET parameters;
To configure STS, see Create custom authorization policies
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: local-volume-provisioner
namespace: kube-system
labels:
app: local-volume-provisioner
spec:
selector:
matchLabels:
app: local-volume-provisioner
template:
metadata:
labels:
app: local-volume-provisioner
spec:
hostPID: true
hostNetwork: true
serviceAccountName: admin
containers:
- image: registry.cn-hangzhou.aliyuncs.com/plugins/local-volume-provisioner:v1.12-7802d35-aliyun
imagePullPolicy: "Always"
name: provisioner
securityContext:
privileged: true
env:
- name: MY_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: ACCESS_KEY_ID
value: ""
- name: ACCESS_KEY_SECRET
value: ""
volumeMounts:
- mountPath: /etc/provisioner/config
name: provisioner-config
readOnly: true
- mountPath: /mnt/disks
name: local
mountPropagation: "HostToContainer"
- mountPath: /etc/kubernetes
name: etc
volumes:
- name: provisioner-config
configMap:
name: local-provisioner-config
- name: local
hostPath:
path: /mnt/disks
- name: etc
hostPath:
path: /etc/kubernetesConsume LocalVolume
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: myclaim
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
storageClassName: local-volume
kind: Pod
apiVersion: v1
metadata:
name: mypod
spec:
containers:
- name: myfrontend
image: nginx
volumeMounts:
- mountPath: "/data"
name: mypd
volumes:
- name: mypd
persistentVolumeClaim:
claimName: myclaim