Deploy Virtual Nodes Quickly with Container Service for Kubernetes
Diverse Combinations of Containers and Serverless
With the popularization of containers and the serverless concept, the Alibaba Cloud Container Service team and the Elastic Compute Service team worked collaboratively and released multiple Serverless container products and solutions, including Serverless Kubernetes, Elastic Container Instance (ECI), and the virtual-nodes add-on feature. These products support a variety of serverless container application scenarios.
Serverless Kubernetes, released by the Alibaba Cloud Container Service team, has been in the open beta test for a while now. Users don’t need to manage servers and can normally perform various Kubernetes API operations, including creating pods, services, ingresses, and jobs. The biggest advantage of Serverless is that it not only avoids the server maintenance burden but also provides applications with powerful elastic scaling. Elasticity is no longer limited to the node scale of clusters. In addition, application billing is based on the resource usage.
Kubernetes Virtual Nodes
We released the virtual-nodes feature to further optimize the Container Service user experience, support more application scenarios and provide extremely powerful elastic scaling for users’ Kubernetes clusters.
As shown in the preceding diagram, this virtual node feature adds virtual nodes to classic Kubernetes clusters, allowing the real worker nodes and virtual nodes to exist in these clusters at the same time. Virtual nodes are used to dynamically create ECI container instances. Pods on ECI and pods on the real nodes are interconnected. They run on the same user’s VPC network.
Virtual-nodes allows users to obtain “unlimited” elasticity in the original Kubernetes clusters without having to create new clusters. Users can also have “unlimited” computing capacity without creating a large number of real worker nodes. The typical application scenarios of virtual nodes include big data computing, gene computing, video rendering, and hybrid clouds. Virtual nodes are also suitable for temporary scaling of online applications.
With virtual nodes, we can implement “unlimited” scaling based on Kubernetes clusters with limited nodes.
How do we use virtual nodes? Let’s first understand its implementation principle.
Elastic Container Instance and Virtual Kubelet
Elastic Container Instance (ECI) is an agile and secure Serverless container runtime service recently released by the Alibaba Cloud ECS team. ECI has the following features:
- Infrastructure management is not required to run containers.
- Flexible on-demand usage scenarios and per-second billing are available.
- The sandbox security technology ensures the application security.
Unlike Serverless Kubernetes Container Service, ECI doesn’t provide the native Kubernetes API. For example, Kubectl cannot be used to perform operations on pod/service/ingress/job resources. However, the innovative connection between Kubernetes and ECI by using virtual kubelet enables ECI instances to be created dynamically by using the Kubernetes API.
Virtual Kubelet is an open source project originally launched by Microsoft Azure with the aim to provide better product integration on public clouds (like elastic containers) with Kubernetes and implement the serverless feature. From the implementation perspective, Virtual Kubelet provides a mechanism that allows integrating products from multiple different providers. Currently, integration with Azure ACI, AWS Fargate, and Huawei CCI is supported. VK is also applied in IoT Edge scenarios. Recently we submitted the ECI provider, which has been merged on https://github.com/virtual-kubelet/virtual-kubelet/tree/master/providers/alicloud?spm=a2c41.12761804.0.0.60f5732bH7UNPS
Virtual Kubelet registers a virtual node with Kubernetes APIServer, continuously listens to pod change events and dynamically creates ECI instances.
Virtual Kubelet innovatively connects Kubernetes and ECI by using virtual nodes. Virtual Kubelet and ECI are the technology infrastructure of virtual nodes.
The following section describes how to deploy virtual nodes in Alibaba Cloud Container Service.
Deploy Virtual Nodes in Alibaba Cloud Managed Kubernetes Clusters
1. Enable ECI
Visit the ECI console and enable ECI.
2. Create a Managed Kubernetes Cluster
Go to the Container Service console and create a Managed Kubernetes cluster:
After a cluster is created, you can view vswitchid and securitygroup in the cluster management or node page.
3. Use kubectl to Deploy Virtual Nodes in One Click
Replace the ECI_VSWITCH, ECI_SECURITY_GROUP, ECI_ACCESS_KEY, and ECI_SECRET_KEY variables in the alicloud-virtual-kubelet.yaml file.
# cat alicloud-virtual-nodes-eci.yaml
- kind: ServiceAccount
- name: alicloud-virtual-kubelet
args: ["--provider", "alibabacloud"]
- name: KUBELET_PORT
- name: VKUBELET_POD_IP
- name: VKUBELET_TAINT_KEY
- name: VKUBELET_TAINT_VALUE
- name: VKUBELET_TAINT_EFFECT
- name: ECI_REGION
- name: ECI_VSWITCH
- name: ECI_SECURITY_GROUP
- name: ECI_ACCESS_KEY
- name: ECI_SECRET_KEY
Execute the yaml file and view the pod and node status. The result shows that a new virtual node has been added to the cluster.
# kubectl apply -f ./alicloud-virtual-nodes-eci.yaml
serviceaccount "alicloud-virtual-nodes-eci" created
clusterrolebinding "alicloud-virtual-nodes-eci" created
deployment "alicloud-virtual-nodes-eci" created
# kubectl -n kube-system get pod|grep virtual-nodes
alicloud-virtual-nodes-eci-54b748489c-gkcmn 1/1 Running 0 53s
# kubectl get node
NAME STATUS ROLES AGE VERSION
cn-hangzhou.i-bp19cyop0b1skne04djt Ready <none> 10m v1.11.2
cn-hangzhou.i-bp19cyop0b1skne04dju Ready <none> 10m v1.11.2
cn-hangzhou.i-bp19cyop0b1skne04djv Ready <none> 10m v1.11.2
virtual-kubelet Ready agent 55s v1.11.2
In the console, we can also see the presence of this virtual-kubelet virtual node.
4. Deploy a Pod to the Virtual Node
Now that a virtual node is present, we need to set nodeSelector and tolerations for the pod to be run and schedule the pod onto the virtual node “virtual-kubelet”.
# cat nginx.yaml
- image: nginx
- key: virtual-kubelet.io/provider
# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx 1/1 Running 0 1m
We can see that the nginx pod is scheduled onto the virtual node and is not running on any worker nodes in the cluster. By using virtual nodes, we implement the serverless deployment of container applications and elastic scaling. Users no longer need to worry about selecting a node scale for clusters or making capacity plans.
Users can follow these steps to quickly deploy virtual nodes. However, there is some complexity because users need to configure AK and vswitch/security groups for clusters. In the future, we will add an Alibaba Cloud Container Service console plug-in for the one-click deployment of virtual nodes. This plug-in will save our users from configuring virtual nodes and provide k8s clusters with powerful elastic scaling. Please stay updated on the latest Alibaba Cloud Container Service news.
- Virtual Kubelet Provier for ECI: https://github.com/virtual-kubelet/virtual-kubelet/tree/master/providers/alicloud?spm=a2c41.12761804.0.0.60f5732bH7UNPS
- Create Managed Kubernetes clusters: https://cs.console.aliyun.com/?spm=a2c41.12761804.0.0.60f5732bH7UNPS#/k8s/cluster/create/managed
- Create Serverless Kubernetes clusters: https://cs.console.aliyun.com/?spm=a2c41.12761804.0.0.60f5732bH7UNPS#/k8s/cluster/create/serverless
- Serverless Kubernetes examples: https://github.com/AliyunContainerService/serverless-k8s-examples?spm=a2c41.12761804.0.0.60f5732bH7UNPS