Practical Exercises for Docker Compose: Part 5

By Alwyn Botha, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud’s incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.

This set of tutorials focuses on giving you practical experience on using Docker Compose when working with containers on Alibaba Cloud Elastic Compute Service (ECS).

Part 4 of this series looked at some productivity tips and best practices of running Docker compose limits. In the final part of this series, we will talk about parallelism in Docker containers and conclude our tutorial series.

Deploy: update_config: parallelism

Parallelism configures how quickly a service should be updated. Containers started up fresh / new are started as quickly as possible. New containers do not use this update_config settings.

Therefore to play with this and test it, we need a set of already running containers so that we can observe the UPDATE parallelism process.

First off parallelism = 1 ( the default ).

Add the following to your docker-compose.yml using

Run:

And around 3 seconds later, we now have 6 containers running. This first docker stack deploy we ran ignored the parallelism option.

Unfortunately, if we just rerun docker stack deploy, we still will not see the parallelism option in action. Docker is too clever — it reads the docker-compose.yml file and see that nothing changed, so it does not update and containers in our mystack.

To prove this, start another console command session and run docker events

Back at the original shell, run 3 times

Observe the other console output:

Expected output :

The service gets updated 3 separate times but no stopping and no starting of fresh containers.

Therefore to force an update we need to make a change to docker-compose.yml

Easiest change: just add a digit to back of the sleep 600.

Do this now, and rerun

Observe the other console output — not shown here.

LOTS of activities: new containers created and old ones killed.

If you run docker ps -a, you will see the new containers at the top and the exited ones below. See that I changed sleep to 6001.

We just witnessed container updates happening. However, the purpose of this part of the tutorial is to show how parallelism works:

Stop previous docker events command in your other console and enter this:

Make any digit change to sleep time in docker-compose.yml

Rerun:

Repeatedly run docker ps -a in first console. After a minute the update process will be done.

You get about one new container every 10–15 seconds. This is what you get with parallelism = 1.

Investigate the second console output: docker events

You will see: 1 new container created, 1 old container killed — and so on for all 6 containers.

To speed this update process up, lets increase parallelism to 3 in our docker-compose.yml

parallelism: 3

Rerun

If you run docker ps -a you will see the new containers at the top and the exited ones below. See that I changed sleep to 6003.

You will immediately see 3 containers created at the top and the remaining 3 created 15 seconds later.

So surely the perfect ideal is parallelism: 6 ? Update all 6 in one go.

Unfortunately, this is not the case. If you have a failing update you will have 6 failed new containers and 6 previous perfectly running containers have all exited.

Fortunately max_failure_ratio and failure_action deals with such problem cases. We’ll discuss more about this topic in the next section.

Deploy: update_config: failure_ratio and failure_action

You can use these 2 settings to lessen the damage caused by failed updates.

Right now we still have our previous containers running. We are going to update that with 6 new containers that each fail immediately.

Add the following to your docker-compose.yml using

Note the command is exit 1 — meaning exit with error response code 1.

Default failure_action is pause but we include it anyway.

Use your other console command session and run docker events

Deploy these failing containers:

Run docker ps -a — after around 30 seconds you will see this:

6 new containers are created at top of list, but 6 old containers are still running below. But since new containers are not running — they are in created state only.

Observe the events console — Docker is continually destroying and recreating these new containers in order to get them to be ‘running’. Clearly pause does not pause.

Cleanup all those containers: ( prune deletes containers that stack rm missed. Yes that happens often unfortunately. )

Pause does not work as I expected. Let’s test failure_action: rollback

To test failure_action: rollback we need to start fresh:

  1. create 6 working containers
  2. create docker-compose.yml that contains 6 exit = 1 error containers
  3. set failure_action to rollback
  4. run deploy stack and observe results.

Add the following to your docker-compose.yml using

Deploy the working containers:

To create 6 error containers: Add the following to your docker-compose.yml using

Deploy the error containers:

Observe results: run docker ps -a repeatedly for around 30 seconds:

Carefully observe the first 3 containers to see if you can determine what is happening.

Here is my final results:

First the new container is created:

Around 5 seconds later one old container is exited.

Then Docker determines the new container cannot progress into running state so it recreates / rolls back one container with the previous settings in docker-compose.yml

This container is listed right at the top. See status: up 22 seconds; it is working.

Slowly scroll through your output and see if the same thing happened on your computer.

Rollback works great.

Conclusion: the settings below is useful in production.

  1. Only update one container at a time.
  2. Tolerate no failures.
  3. Rollback on failure.

Deploy: update_config: monitor

From https://docs.docker.com/compose/compose-file/#update_config

monitor: Duration after each task update to monitor for failure (ns|us|ms|s|m|h) (default 0s).

There are no exercises here, but this setting is important.

As you just read its the duration after container update to monitor for failure.

Your application in the container must have sufficient time to do some work before Docker can test it for failure.

The default duration value of 0 seconds is not suitable for practical purposes. This is useful only to ensure that the container start up successfully.

Let’s assume you run a web server inside the container. Further assume one website visit per minute. So to have monitor set at 1 minute is probably too frequent. Instead, you could set it at least 5 minutes — to give it opportunity to crash before you test for failure.

You must determine an appropriate monitor duration value for your applications at your workplace.

Health checks are similar to asking the question: can I ping it?

These monitor for failure tests if the container as a whole crashes or not. They test whether the application in the container still work in the context of the other containers that it must cooperate with.

For example, if you deploy a new Apache version, but fail to ensure it can load all required modules, health check pings and wgets of text html pages will work perfectly, but the moment php + Mysql pages need to be served it will crash.

That is why you should give your application in the newly updated container sufficient time to run before Docker can test it for failure.

We want to develop health checks that not only test connectivity, but also functionality.

Such health checks can then within seconds indicate an unhealthy new container — minutes before it gets ( and fails ) at processing real production work.

Parallelism Experiments Output

Below is some docker events output I gathered by experimenting with different parallelism settings: ( I output the events to /tmp and manipulated that file afterwards )

Two kill lines are for one container: signal 15; if not dead quickly enough: send signal 9

parallelism: 1

parallelism: 2

parallelism: 3

parallelism: 6

Deploy: restart_policy

This configuration option specify how to restart containers when they exit.

The default value for condition is any: this means that container restarts on-failure or when it successfully exits.

restart_policy is applied only when deploying a stack in swarm mode.

restart is applied when deploying using docker-compose up — starting a single service.

To see how this works, we will successfully exit our container after a 3 seconds sleep. We expect the container to be restarted automatically.

Add the following to your docker-compose.yml using

After running docker ps -a repeatedly for 25 seconds you will see the trend: container exits after 3 seconds and new one is automatically created to fill its place. The default restart_policy works as expected.

Expected output after around 25 seconds:

This will go on forever. Fortunately we can limit this restart max attempts.

max_attempts specifies how many times to attempt to restart a container before giving up (default: never give up)

Add the following to your docker-compose.yml using

Deploy this using:

If you run you will notice via repeated docker ps -a after 20 seconds that only 3 new containers get created. Then the auto restarting stops as expected.

You can also adjust how long to wait between restart attempts using delay.

Its default value is zero as you have seen: restarts happen immediately.

You can set this to any duration you desire. This tutorial will not cover this, but you are welcome to perform a quick test.

Finally there is also a window option:

window: How long to wait before deciding if a restart has succeeded, specified as a duration (default: decide immediately).

Feel free to test this on your own as well.

For an even more interesting exercise experiment with the interaction between delay and window.

Conclusion

This concludes this tutorial set, but only marks the start of your docker-compose training on Alibaba Cloud Elastic Compute Service (ECS). You should now have a solid understanding and considerable practical experience of some docker-compose configuration options.

To learn more about Docker compose, visit the official documentation at https://docs.docker.com/compose/compose-file/

Or check out Alibaba Cloud’s Container Service to learn how you can reap the full benefits of containerization.

Reference:https://www.alibabacloud.com/blog/practical-exercises-for-docker-compose-part-5_594421?spm=a2c41.12541340.0.0

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store