Getting Started with Kubernetes | Service Discovery and Load Balancing in Kubernetes
By Wang Bingshen (Xiheng), Technical Expert at Alibaba
1. Source of Requirements
Purpose of Service Discovery
Applications in a Kubernetes cluster are deployed by using pods, which is different from the conventional approach of deploying applications on specific machines. We know how to call the IP addresses of other machines, but the pod lifecycle is short. The IP address of a pod changes during its lifecycle, for example, when the pod is created or destroyed. This makes it impossible to deploy applications in a Kubernetes cluster by using the conventional approach because application access through a specified IP address is impossible.
To deploy applications in a Kubernetes cluster, you need to create a group of pods in addition to Deployments, provide a central endpoint for the pods, and implement traffic load balancing among the pods. You need to keep the same deployment template and access mode for the test environment, staging environment, and production environment when deploying applications. In this way, you can use the same template to directly publish applications to different environments.
Kubernetes Services: Service Discovery and Load Balancing
Applications need to be exposed so that external users can call them. Pods are located in a different network segment from machines. To expose the pod network for external access, you need to configure service discovery.
In Kubernetes, service discovery and load balancing are provided as services. The preceding figure shows the architecture of a Kubernetes Service, which provides access to the external network and pod network in the uplink direction.
The Kubernetes Service interworks with a group of pods in the downlink direction to implement load balancing among the pods. This provides a central endpoint for service discovery and enables access to the external network and access among different pods through the same address.
2. Case Study
The following is an actual use case to show how to declare and use Services for pods in Kubernetes.
To start, let’s look at the syntax of a Kubernetes Service. The preceding figure shows a declaration structure of Kubernetes. This structure contains many syntaxes, which are similar to some standard Kubernetes objects described earlier. For example, you can make selections by using the labels and selector fields and make declarations by using the labels field.
The template defines a protocol and a port for the Kubernetes Service used for service discovery. The template declares a Kubernetes Service named
my-service, which has the
app:my-service label. The Service selects the pod with the
app:MyApp label as its backend.
TCP and port
80 are defined for service discovery. The target port is
9376. Access requests to port
80 are routed to port
9376 of the pod with the
app:MyApp label at the backend. This implements load balancing.
Create and View a Service
This section explains how to create the Service declared earlier and view the created Service. Run the following command:
kubectl apply -f service.yaml
kubectl created -f service.yaml
The preceding commands are used to create a Service. After a Service is created, run the following command:
kubectl describe service
This lets you view the created Service.
The created Service is named
Selector fields are the same as the fields in the earlier declaration. An IP address is created for the Service and can be accessed by pods in the cluster. The IP address is the central endpoint for all pods and is used by service discovery.
Endpoints field indicates the pods specified by the
selector field. You can view the pod status. For example, you can view the IP addresses and target ports of the selected pods.
The preceding figure shows the architecture. A virtual IP address and a port are created along with the Service in the cluster. All pods and nodes in the cluster can access the Service through this IP address and port. The Service mounts the selected pods and their IP addresses to the backend. In this way, access requests through the Service’s IP address are distributed to these pods for load balancing.
When a pod’s lifecycle changes, for example, when the pod is destroyed, the Service automatically removes the pod from the backend. This ensures that the endpoint remains unchanged despite the changes in the pod lifecycle.
Service Access Within a Cluster
After a Service is created in a cluster, the pods in the cluster can access the Service in one of the following ways:
- The pods can access the Service through its virtual IP address. For the Service named
my-servicecreated earlier, you can run the
kubectl get svcor
kubectl describe servicecommand to view the virtual IP address
80of the Service, which can be directly used by pods to access the Service.
- The pods in the same namespace as the Service can access the Service through its name after the name is resolved by DNS. The pods in a different namespace from the Service can access the Service in the format of
service_name.service_namespace. For example, the Service can be accessed through curl in the format
- The pods in the same namespace as the Service can access the Service through the environment variables that are used to transfer the Service’s IP address, port, and simple configuration to the pods at startup. After the containers of the pods are started, they read the environment variables to retrieve the IP address and port of the Service in the same namespace. For example, a pod in a cluster can use curl
$to get the value of an environment variable.
MY_SERVICE_SERVICE_HOSTindicates the Service's IP address,
MY_SERVICEis the declared service name, and
SERVICE_PORTis the Service's port. In this way, the pod can send requests to the Service indicated by
MY_SERVICEin the cluster.
Headless services are a special type of service. When creating a Service, you can specify
clusterIP:None to tell Kubernetes that you do not need the cluster IP address, that is, the virtual IP address mentioned earlier. Then, Kubernetes does not allocate a virtual IP address to this Service. Even without the virtual IP address, the Service can implement load balancing and provide a central endpoint as follows:
Pods can directly resolve the Service name through the A record of DNS to the IP addresses of all pods at the backend. The client can select any of the resolved backend IP addresses. The A record changes with the changes in the pod lifecycle, as does the returned A record list. The client needs to select an appropriate IP address from the list of the DNS-returned A record to access pods.
Compared with the earlier-declared template, the template in the preceding figure adds
clusterIP:None, indicating that the virtual IP address is not required. When a pod in the cluster accesses
my-service, it directly resolves the service name to the IP addresses of all pods that match the Service. Then, the pod selects an IP address from the returned list to directly access the Service.
Expose a Service Outside the Cluster
The preceding section showed how to access a Service from nodes or pods in a cluster. This section shows how to expose a Service outside the cluster and expose applications for access from the Internet. You can use
LoadBalancer to expose Services externally.
NodePortmode, the port of a node in a cluster is exposed on the node host. When receiving an access request, the exposed port forwards the request to the virtual IP address of the Service configured on the host.
LoadBalancermode, an additional forwarding layer is added.
NodePortis implemented on the port of every node in the cluster, whereas
LoadBalancermounts a load balancer to all nodes. For example, you can mount an Alibaba Cloud Server Load Balancer (SLB) instance to provide a central endpoint and evenly distribute incoming traffic to the pods of all nodes in the cluster. Then, the pods forward the traffic to the target pods based on the cluster IP address.
The following example demonstrates how to use a Kubernetes Service in Alibaba Cloud Container Service.
Create a Service
Prerequisites: You have created an Alibaba Cloud container cluster and configured a connection from the local terminal to the Alibaba Cloud container cluster.
kubectl get cs command to check that the Alibaba Cloud container cluster is connected.
You can use the following templates to implement a Kubernetes Service in Alibaba Cloud. There are three templates. The client template is used to access a Kubernetes Service that evenly distributes traffic to the pods declared by the Service.
Create a Kubernetes Service template to declare pods so that traffic is evenly distributed from port
80 at the frontend to port
80 at the backend. Then, set the
selector field to
select backend pods with the
Then, create a group of pods with the
run:nginx label by using Kubernetes Deployments. A Deployment has two replicas, matching two pods.
kubectl create -f service.yaml command to create a Deployment. After a Deployment is created, check whether pods are also created. As shown in the following figure, the two pods created along with the Deployment are in the
Running state. Run the
kubectl get pod -o wide command to view the IP addresses of the pods. Use
-l to implement filtering based on
run=nginx. As shown in the following figure, the two pods have the IP addresses
10.0.0.12, and both have the label
Run the following command to create a Kubernetes Service that selects the two pods:
kubectl describe svc command to view the status of the Service. As shown in the following figure, the created Kubernetes Service named
nginx uses the selector
run=nginx to select the pods
10.0.0.135 as backend pods. A virtual IP address in the cluster is created for the Kubernetes Service to evenly distribute traffic to the two pods at the backend.
client.yaml command to create a client pod to access the Kubernetes Service. Run the
kubectl get pod command to check that the client pod was created and in the
kubectl exec command to access the client pod and experience the three access modes. Use curl to directly access the cluster IP address (or virtual IP address) of the Kubernetes Service. The client pod does not have curl installed. Run the
wget command and enter the virtual IP address. You can access the Kubernetes Service named
nginx at the backend through the virtual IP address, which is also the central endpoint.
You can also access the Kubernetes Service through the service name. Run the
wget command to access the Kubernetes Service
nginx and you will get the same result as earlier.
If the client pod is in a different namespace from the Kubernetes Service, you can add the name of the namespace where the Service is located to access the Service. Here, we use the namespace named
default as an example.
You can also access the Kubernetes Service through environment variables. Run the
env command on the client pod to view the injected environment variables. All configurations of the
nginx Service are registered.
wget command to access the environment variables. Then, you can access the Kubernetes Service.
The following explains how to access the Kubernetes Service from an external network. Modify some configurations of the Kubernetes Service in
type field and set it to
LoadBalancer to enable external access.
kubectl apply command to apply the modifications to the Service.
Now, let’s see what changes occur in the Service. Run the
kubectl get svc -o wide command and you will find that the Service
EXTERNAL-IP, which is the IP address for external access. As mentioned earlier, the Service is accessed within the cluster through the virtual IP address defined by
Access the external IP address
220.127.116.11 to see how applications are exposed through the Service. Enter the external IP address in the web browser of the terminal to access the Service.
The following shows how to use the Service to implement service discovery in Kubernetes. The Service access address is unrelated to the pod lifecycle. Let’s first look at the IP addresses of the two pods selected for the Service.
kubectl delete command to delete the first pod.
Then, the Deployment automatically creates another pod with an IP address that ends with
Run the describe command to view the Service information, as shown in the following figure. The endpoint is still the cluster IP address. In
LoadBalancer mode, the IP address for external access remains unchanged. The IP address of a backend pod is automatically included in the backend IP address list of the Service. This does not affect client access.
In this way, the changes in the pod lifecycle have no impact on calls to application components.
4. Architecture Design
This chapter analyzes the design and implementation of Kubernetes.
Kubernetes Service Discovery Architecture
The preceding figure shows the architecture of a Kubernetes Service for service discovery.
The architecture contains a master node and multiple worker nodes.
- The master node implements the control function in Kubernetes.
- The worker nodes run user applications.
The Kubernetes API Server is deployed on the master node to centrally manage all Kubernetes objects. All components are registered on the API server to listen to object changes, such as changes in the pod lifecycle.
The master node has three major components:
- The cloud controller manager configures a load balancer for external access.
CoreDNSlistens to changes of the backend pods of the Service on the API server. You can configure DNS resolution to directly access the virtual IP address of the Service through the service name. You can also configure DNS resolution to resolve the IP addresses in the IP address list kept by the headless Service.
kube-proxyon every node listens to changes of the Service and pods, allowing you to configure the nodes and pods in the cluster or configure access through the virtual IP address based on the actual situation.
Let’s have a look at the actual access link. For example, assume Client Pod 3 in the cluster wants to access the Service. Client Pod 3 resolves the Service IP address through
CoreDNS, which returns the IP address that matches the service name. Client Pod 3 initiates a request through the Service IP address. After the request is sent to the host network, it is intercepted based on iptables or the IP virtual server (IPVS) configured by
kube-proxy. Then, the request is distributed by the load balancer to each pod at the backend. This implements the service discovery and load balancing processes.
Let’s look at how external traffic is processed, for example, when a request is sent from the Internet. A load balancer is configured after the external cloud controller manager, which is also a load balancer, listens to changes of the Service. The configured load balancer forwards an external access request to the port of a node, which then forwards the request to the cluster IP address based on the iptables configured by
kube-proxy. Then, the cluster IP address is mapped to the IP address of a backend pod, to which this request is finally sent. This implements the service discovery and load balancing processes. This is the architecture of a Kubernetes Service for service discovery.
This section explains how to implement a Kubernetes Service and how to diagnose and fix network errors of the Service.
Let’s summarize what we have learned in this article:
- The purposes of service discovery and load balancing in cloud-native scenarios
- How to implement a Kubernetes Service for service discovery and load balancing
- What components in a Kubernetes cluster are used by the Service and how these components work.
I hope that, after reading this article, you can orchestrate complex enterprise-level applications in a standard and fast manner by using Kubernetes Services.