Traffic Management with Istio (3): Traffic Comparison Analysis based on Istio

Join us at the Alibaba Cloud ACtivate Online Conference on March 5–6 to challenge assumptions, exchange ideas, and explore what is possible through digital transformation.

Traffic Mirroring

Traffic mirroring, also known as traffic shadowing, provides a powerful way to bring changes to production at the lowest possible risk. The mirror sends a copy of real-time traffic to the mirroring service. Mirrored traffic goes outside of the critical request path of the main services.

In non-production or test environments, trying to access all possible combinations of test cases for a service is unrealistic. In some cases, the work of writing these test cases may not match actual production needs. In the ideal case, you can use real-time production use and traffic to help improve the functional regions you miss in the test environment.

Once we are able to reliably mirror traffic, we can start other valuable tasks. For example, using Diffy, a request traffic comparison tool, we can compare the traffic of the introduced test cluster to the expected behavior of the production cluster. For example, we might want to compare the deviation between the request results and the expected results, or data corruption in the API Protocol, for better compatibility.

In addition, please note:

  1. When the traffic is mirrored to a different service, it occurs outside the critical path of the request.
  2. Ignore response to any mirrored traffic. This traffic is considered to be “instantly forgotten”.

Traffic Comparison

Here, by inserting a proxy, you can be responsible for the coordination of such traffic, and it makes an interesting comparison. Diffy is such a proxy tool. Diffy starts a proxy service (listening, for example, on port 8880 ), again, based on the primary and secondary old service addresses set by the user, (the primary and secondary codes are identical and the purpose is to reduce noise interference) and a new candidate service address.

It can also detect noise in the result, and ignore instances of two real-time services by first calling them (for example, timestamps, monotonically increasing counter and other prompts). In summary, it detects and then ignores these parts in the test service.

Diffy also provides a very good page to view the results of the call, and compare the conditions, which can be filtered by particular characteristics It also has a good management console where you can view the metrics and statistics of the comparing call results function.

Creating a Service for Istio Traffic Mirroring

In this task, you first force all traffic to the v1 version of the service. You will then use a rule to mirror a portion of the traffic to the v2 version.

Two versions of the sample service are first deployed.

Docker mirroring httpbin is used to provide common http access requests in the deployment of version 1:

A custom docker image is used in the deployment of version 2, and the corresponding Dockerfile is as follows:

Required nginx configuration files:

Version 2 is deployed to act as the traffic mirror target for Istio. After receiving the traffic, it is forwarded to the Diffy proxy. The Diffy proxy is not currently used as the Istio traffic mirror target. This is because of a conflict between the current versions of the Diffy proxy and the Envoy proxy, making normal traffic forwarding impossible. As a result, this deployment is needed to mediate the traffic.

Corresponding Kubernetes service:

Creating Istio Policy for Traffic Mirroring

By default, Kubernetes performs load balancing between the two versions of the service. Create the following traffic mirroring rule to send 100% of the traffic to v1, and specify that the traffic is mirrored to v2. When the traffic is mirrored, requests will be sent through its host/authorized header to the mirror service with the appended -shadow.

Setting Up Diffy to Request Traffic Comparison

Diffy can be used as a proxy to intercept requests and send them to all instances of the running service. Problems that may exist in each iteration code are identified by comparing the response results. Among them, there are three code instances running on Diffy:

  1. Online stable version: A node that runs a stable version of online code
  2. Online stable version backup: Also runs an online stable version to eliminate noise
  3. Beta version: A beta version awaiting release, for comparison with the code of the online environment

In the actual Diffy test, you will find there is some difference between the majority of interfaces. This is because of noise in the responses, including:

  1. Timestamps generated in the server response
  2. Randomly generated numbers
  3. Conditional competition among system services

Diffy can eliminate such noise to ensure the results of the analysis are not affected.

Creating a Diffy and Sample Service

Create the Diffy service with the following yaml:

Create the primary, secondary (same as the primary in the current sample) and candidate services used in the sample with the following YAML:

Send Traffic for Mirror Verification

Start the sleep service so you can use curl to provide the load:

Enter into SLEEP_POD. The specific pod name will vary according to the actual assignment.

Send traffic:

Check the access log for v1. As shown below, 100% of the requests created were for v1.

Also, if you check the Diffy web interface, you can see that the created requests were also mirrored to the Diffy proxy.

Diffy can eliminate such noise to ensure the results of the analysis are not affected.


Traffic mirroring offers powerful features that bring changes to production with as little risk as possible. Mirroring sends a copy of live traffic to a mirrored service. The mirrored traffic occurs outside the critical request path of the primary service. Once we are able to reliably mirror traffic, we can start doing other valuable tasks. For example, using Diffy — a request volume comparison tool — we can compare the traffic of the introduced test cluster to the expected behavior of the production cluster.

Supporting traffic mirroring is just one of Istio’s numerous features which will simplify the production deployment and management of large microservice-based applications. We invite you to use Alibaba Cloud Container Service to quickly set up Istio, an open management platform for microservices that can be more easily integrated into any microservice projects you are working on.


Follow me to keep abreast with the latest technology news, industry insights, and developer trends.