North-South Traffic Management of Istio Gateways (with Answers from Service Mesh Experts)

Image for post
Image for post

By Wang Xining, Senior Technical Expert at Alibaba Cloud

Istio Gateway

The term “ingress” in the Internet community refers to the management of traffic from external access requests to services in a cluster. Ingress refers to the traffic originating from outside the local network and pointing to endpoints in the local cluster network. The traffic is first routed to public entries so that some local network rules and policies are enforced to determine which traffic is allowed to pass through. If traffic is not allowed to pass through these entries, it cannot connect to any services in the cluster. If the entries allow traffic to enter, they route the traffic to the appropriate nodes in the local network through proxies. An Istio gateway manages the ingress traffic.

How an Istio Gateway Works

Traditionally, Kubernetes uses Ingress controllers to handle traffic from the outside to the cluster. This is no longer the case while using Istio. Istio gateways use new Gateway resources and VirtualServices resources to control ingress traffic. They work together to route traffic to the service mesh. Gateways are not required within the service mesh because services access each other based on local service names in the cluster.

Then, how does an Istio gateway work? How does a request reach its target application? The basic steps are as follows:

1) The client sends a request on a specific port.
2) The Server Load Balancer (SLB) listens to this port and forwards the request to the cluster (on the same port or another port).
3) Within the cluster, the request is routed to the port forwarded by the SLB which was listened to by the Istio IngressGateway service.
4) The Istio IngressGateway service forwards the request (on the same port or another port) to the corresponding pod.
5) Gateway resources and VirtualService resources are configured on the IngressGateway pod. The port, protocol, and related security certificates are configured on the Gateway. The VirtualService routing information is used to find the correct service.
6) The Istio IngressGateway pod routes the request to the corresponding application service based on the routing information.
7) The application service routes the request to the corresponding application pod.

Image for post
Image for post
Figure 1: How an Istio gateway works

Load Balancing of Istio Gateways

A typical service mesh has one or more SLBs, also known as Gateways, which terminate transport layer security (TLS) connections from the external network and allow traffic to enter the mesh. Then, the traffic passes through the Sidecar gateway to the internal services. It is also common for applications to use external services. Applications directly call external services or, in some deployments, force traffic away from the mesh through dedicated egress gateways.

Istio has features of an ingress gateway. It acts as a network entry and is responsible for protecting and controlling the access from external traffic to the cluster.

Image for post
Image for post
Figure 2: Gateway usage in mesh

In addition, an Istio gateway performs load balancing and virtual host routing. As shown in the figure, by default, Istio uses the Envoy proxy as the ingress proxy. Envoy is a powerful service-to-service proxy. It also has load balancing and routing functions. The proxy traffic includes services running from outside the service mesh to inside, or services from inside a cluster to outside. The Envoy functions described in the previous section may also be provided by the ingress gateway.

Image for post
Image for post
Figure 3: Ingress gateway service of Istio

You may ask, why not just use the Kubernetes Ingress APIs to manage ingress traffic?

The Istio gateway resolves these problems by separating L4-L6 configurations from L7 configurations. Only L4-L6 functions (for example, exposed ports and TLS configurations) are configured on Istio gateways. These functions are implemented in a unified manner by all mainstream L7 proxies. Then, by binding VirtualService to a gateway, use standard Istio rules to control HTTP and TCP traffic entering the gateway. SLB can be manually configured or its type can be automatically configured through the service, for example, type: LoadBalancer. In this case, since not all clouds support automatic configuration, assume that SLB is manually configured to forward traffic to the port to which the IngressGateway Service is listening. For example, SLB is listening to the following ports:

Make sure that the SLB is configured to forward traffic to all work nodes. This ensures that traffic is forwarded even when some nodes are shut down.

Ingress Gateway Service

The IngressGateway service must listen to all the ports described in the previous section, so that it forwards traffic to IngressGateway pods. The Kubernetes service is not a “real” service. A request is forwarded to the node that runs the corresponding pod by kube-proxy. Kube-proxy is provided by Kubernetes. The node forwards the request to the appropriate pods based on the IP table configuration.

ports:
- name: http2
nodePort: 30000
port: 80
protocol: TCP
- name: https
nodePort: 30443
port: 443
protocol: TCP
- name: mysql
nodePort: 30306
port: 3306
protocol: TCP

Ingress Gateway Deployment

IngressGateway is deployed as an encapsulation based on the Envoy proxy. IngressGateway configuration is the same as the Sidecar configuration used in the service mesh. They are actually the same container image. While creating or modifying a gateway or VirtualService, the Istio Pilot controller detects the changes and converts the changes into Envoy configurations. The Envoy configurations are then sent to the relevant Envoy proxies, including the internal Envoy and the Envoy in IngressGateway.

Note: Do not confuse IngressGateway with a gateway. Gateway resources are custom Kubernetes resources used to configure IngressGateway.

There is no need to declare a container port in a Kubernetes pod or deployment. Therefore you do not need to declare a port in IngressGateway deployment either. However, on looking inside the deployment, you may see many declared ports. In addition, ensure SSL certificates are correctly loaded when IngressGateway is deployed so that they are easily accessed.

Gateway Resources

Gateway resources are used to configure the Envoy port. In the preceding example, the IngressGateway service is used to expose three ports. Therefore, you must resolve these ports in Envoy. In addition, one or more gateways can be declared to support the multi-port capability. The following example uses a single gateway, but it can be defined in two or three pieces:

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: default-gateway
namespace: istio-system
spec:
selector:
istio: ingressgateway
servers:
- hosts:
- '*'
port:
name: http
number: 80
protocol: HTTP
- hosts:
- '*'
port:
name: https
number: 443
protocol: HTTPS
tls:
mode: SIMPLE
privateKey: /etc/istio/ingressgateway-certs/tls.key
serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
- hosts: # For TCP routing this fields seems to be ignored, but it is matched
- '*' # with the VirtualService, I use * since it will match anything.
port:
name: mysql
number: 3306
protocol: TCP

Gateway Virtual Service

VirtualService resources and Gateway resources work together to support the configuration of Envoy. The following is the basic configuration of a gateway VirtualService that supports HTTP services:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: counter
spec:
gateways:
- default-gateway.istio-system.svc.cluster.local
hosts:
- counter.lab.example.com
http:
- match:
- uri:
prefix: /
route:
- destination:
host: counter
port:
number: 80

Now, when we add a gateway and a VirtualService, the route is already created in the Envoy configuration. Use the following command to view this content:

kubectl port-forward istio-ingressgateway-xxxx-yyyy-n istio-system 15000

Debugging Ingress Gateway

Debugging network problems sometimes becomes difficult. Here are some useful commands for debugging.

Forward traffic to the first Istio-IngressGateway pod:

kubectl -n istio-system port-forward $(kubectl -n istio-system get pods 
-listio=ingressgateway -o=jsonpath="{.items[0].metadata.name}") 15000

Then, obtain the HTTP route from the IngressGateway pod forwarded by the port:

Curl --silent http://localhost:15000/config_dump |jq .configs[3].dynamic_route_configs[].route_config.virtual_hosts[]
Image for post
Image for post
Figure 4: IngressGateway pod forwarded by the port

View the logs of the IngressGateway pod forwarded by the port:

kubectl -n istio-system logs $(kubectl -n istio-system get pods 
-listio=ingressgateway -o=jsonpath="{.items[0].metadata.name}") --tail=300

View the logs of the Pilot pod:

kubectl -n istio-system logs $(kubectl -n istio-system get pods 
-listio=pilot -o=jsonpath="{.items[0].metadata.name}") discovery --tail=300

After the boot port forwards information to Istio-IngressGateway, perform more operations to obtain more information. For example:

About the Author

Wang Xining, senior technical expert at Alibaba Cloud, and technical owner of ASM and Istio on Kubernetes for Alibaba Cloud, specializes in Kubernetes, cloud-native, service mesh, and other fields. Earlier, Wang worked with IBM China Development Center and served as the chairman of the patent technology review committee. He holds more than 40 international technology patents in related fields. “Technical Analysis and Practice of Istio Service Mesh” introduces the basic principles and development practices in detail, and selects abundant cases and reference codes for download, to help users get started with Istio development quickly. Gartner believes that service mesh will become the standard technology of all leading container management systems in 2020. His book is suitable for all readers who are interested in microservices and cloud-native, and we recommend in-depth reading of this book.

Q&A About Istio

Q1) What are the bottlenecks Istio encounters in practical production environments and what are the common optimization methods to tackle them?

A1) Alibaba Cloud launched ASM for service mesh. In the last several years, we have encountered a lot of problems while assisting customers in using Istio. We have summarized our experience and put it into ASM. So please check out the abilities of this product, and explore what problems it solves. I will not go into detail here.

Q2) Does Istio increase performance consumption?

A2) This question requires more context, as it is like asking whether Java Virtual Machine causes performance consumption. Any decoupling will always bring a certain amount of communication consumption. We recommend determining whether your applications are suitable for decoupling, servitization, and containerization before you use Istio.

Q3) Service Mesh is a very good tool, but it might also be difficult to use. The introduction of Istio will make operation and maintenance more complex. What improvements has Alibaba Cloud made for ASM in this regard?

A3) The Alibaba Cloud ASM provides a fully-managed service mesh platform that is compatible with the Istio open-source service mesh, to simplify service governance, and provide a simple and easy-to-use console. The hosted mode frees users from complex control-plane management and greatly reduces the workload of development and O&M. For more information, see the introductory tutorial.

Q4) I have been studying Istio recently. Are there any examples of use cases?

A4) In recent years, we have supported a large number of customers using Istio. For example, some customers have implemented a phased release of applications using Istio traffic management. Some customers use Istio’s unified declarative approach to manage Ingress Gateway, including the TLS transparent transmission, TLS termination, and the ability to dynamically load certificates.

Q5) What kind of monitoring services does ASM currently adopt for Istio?

A5) ASM provides observability functions from three dimensions:

Q6) Will the ASM Proxy of Alibaba Cloud use MOSN? I expect MOSN to become one of the optional data planes in Istio.

A6) ASM was initially designed to be compatible with Istio. Theoretically, as long as a data-plane proxy meets Istio control-plane requirements, it is supported. Of course, the adaptation of a new proxy requires a certain amount of development work, and we also want to know the customers’ demands for this aspect.

Q7) Does Istio provide features like mutls from Linkerd?

A7) Istio provides two-way TLS authentication by default and supports progressive authentication using two methods: permissive and strict.

Original Source:

Written by

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store