Practical Exercises for Docker Compose: Part 3

By Alwyn Botha, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud’s incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.

This set of tutorials focuses on giving you practical experience on using Docker Compose when working with containers on Alibaba Cloud Elastic Compute Service (ECS).

Part 2 of this series explored several Docker compose configurations. In Part 3, we will explore depends_on, volumes and the important init docker-compose options.

depends_on

depends_on declares dependencies between services.

depends_on let docker-compose up starts services in dependency order.

The dependent services start first, before the main service starts.

As you will see the dependencies start in random order. Also, docker-compose does not wait for any dependencies to be running before starting the main service.

Let’s see that in action so you can understand how it works:

Add the following to your docker-compose.yml using

nano docker-compose.ymlversion: "3.7"services:alpine:image: alpine:3.8command: sleep 600depends_on:- service-2- service-3- service-4container_name: main-containerservice-2:image: alpine:3.8container_name: service-2command: sleep 600service-3:image: alpine:3.8container_name: service-3command: sleep 600service-4:image: alpine:3.8container_name: service-4command: sleep 600

Please note the dependencies are declared in neat number order.

Let’s start up this docker-compose file using:

docker-compose up -d -t 0

Expected output :

Creating service-3 ... doneCreating service-4 ... doneCreating service-2 ... doneRecreating compose-tuts_alpine_1 ... done

Note that dependencies started in random order. The main service started last — not clearly shown in above output.

If you make small edits to the docker-compose file ( such as sleep time edits ) and redo the docker-compose up you will see those services created in different sequences each time. Try it.

To more clearly see the sequence of events, let us shut down all services an redo the docker-compose up.

docker-compose down -t 0

Expected output :

Stopping main-container ... doneStopping service-4  ... doneStopping service-3  ... doneStopping service-2  ... doneRemoving main-container ... doneRemoving service-4  ... doneRemoving service-3  ... doneRemoving service-2  ... doneRemoving network compose-tuts_default

Note the main-container gets stopped first, then the dependencies.

Now run

docker-compose up -d -t 0

Expected output :

Creating network "compose-tuts_default" with the default driverCreating service-3 ... doneCreating service-2 ... doneCreating service-4 ... doneCreating main-container ... done

Clearly shows main-container created last.

If you need you dependency services to be READY before the main container starts you need to study https://docs.docker.com/compose/startup-order/

volumes

volumes are used to mount named volumes.

To make this interesting we will reuse the 4 services from the previous example.

All 4 services will use the named volume: demo-data-volume. It does not exist yet.

Add the following to your docker-compose.yml using

nano docker-compose.ymlversion: "3.7"services:alpine:image: alpine:3.8command: sleep 600depends_on:- service-2- service-3- service-4container_name: main-containerservice-2:image: alpine:3.8container_name: service-2command: sleep 600volumes:- demo-data-volume:/root/dir1/dir2service-3:image: alpine:3.8container_name: service-3command: sleep 600volumes:- demo-data-volume:/root/dira/dirbservice-4:image: alpine:3.8container_name: service-4command: sleep 600volumes:- demo-data-volume:/root/1/2/3/4volumes:demo-data-volume:

Right at the bottom: that is the top level volumes key ( as named in the Docker documentation ).

Top level keys start at extreme left side of the docker-compose file.

Our top level key defines demo-data-volume — the named volume we want to use.

When we run

docker-compose up -d -t 0

demo-data-volume will be created ( if it does not exist ).

Expected output :

Creating network "compose-tuts_default" with the default driverCreating volume "compose-tuts_demo-data-volume" with default driverCreating service-2 ... doneCreating service-4 ... doneCreating service-3 ... doneCreating main-container ... done

Note second line: Creating volume “compose-tuts_demo-data-volume” with default driver

Run:

docker volume ls

Expected output :

DRIVER  VOLUME NAMElocal  compose-tuts_demo-data-volume

There is our named volume. Its name is the concatenation of our current directory and the volume name we specified in our docker-compose file.

In our docker-compose file we defined that we want to use this named volume at different mount points in 3 services.

service-2:volumes:- demo-data-volume:/root/dir1/dir2service-3:volumes:- demo-data-volume:/root/dira/dirbservice-4:volumes:- demo-data-volume:/root/1/2/3/4

We are now going to see these 3 services all have access to the same named volume.

Let’s enter service-2 and work with our named volume.

docker exec -it service-2 /bin/sh

Execute commands as shown: ls to show the path exists and echo to create myoutfile. Then exit.

/ # ls /root/dir1dir2/ # echo from service 2 > /root/dir1/dir2/myoutfile/ # exit

Let’s enter service-3 and investigate our named volume.

docker exec -it service-3 /bin/sh

Enter commands shown: ls to confirm that our file exists. cat it to show its content.

/ # ls /root/dira/dirbmyoutfile/ # cat /root/dira/dirb/myoutfilefrom service 2/ # exit

Note that with service-3 the volume is mounted at /root/dira/dirb

service-3:volumes:- demo-data-volume:/root/dira/dirb

Note that with service-4 the volume is mounted at /root/1/2/3/4

service-4:volumes:- demo-data-volume:/root/1/2/3/4

Enter service-4 to confirm that our file is available there.

docker exec -it service-4 /bin/sh

Expected output :

/ # cat /root/1/2/3/4/myoutfilefrom service 2/ # exit

Our named volume is available for reading and writing inside these 3 containers. It will continue to exist even if we stop these containers.

Other containers, services and swarms are allowed to used this named volume. It is not specifically linked to these 3 services.

During these few minutes you explored probably less than 5% of Docker volume functionality.

There are more than 25 volume plugins listed athttps://docs.docker.com/engine/extend/legacy_plugins/#volume-plugins

You can read the official docs at https://docs.docker.com/storage/volumes/ and https://docs.docker.com/storage/

At least now you have actually used named volumes and have some idea of how it works.

Fortunately named volumes are the easiest to understand, most flexible and most frequently used.

restart

restart is applied when starting a service using docker-compose up

It has 4 easy to understand options:

  1. restart: “no” … the default value
  2. restart: always
  3. restart: on-failure
  4. restart: unless-stopped

Let’s experiment with those:

Demo for restart: “no” … the default value

Add the following to your docker-compose.yml using

nano docker-compose.ymlversion: "3.7"services:alpine:image: alpine:3.8command: sleep 1

Bring up the service:

docker-compose up -d -t 0

Check its status:

docker ps -a

The sleep 1 causes the container to exit after just one second.

You will only see one exited container — even if you rerun docker ps -a many times. As expected: restart = no does not restart containers.

Demo for restart: unless-stopped

Add the following to your docker-compose.yml using

nano docker-compose.ymlversion: "3.7"services:alpine:image: alpine:3.8command: sleep 1restart: unless-stopped

Prune previous container — so we have a clean docker ps -a to work from.

docker container prune -f;docker ps -a

Bring up new container.

docker-compose up -d -t 0

Check its status:

docker ps -a

The sleep 1 causes the container to exit after just one second.

If you run this command repeatedly you will see the container continually restarting.

docker container stop your-container-id does not stop it.

docker-compose down removes the container within seconds.

If you use docker-compose up to start up a container, use docker-compose down to take it down.

restart: unless-stopped work as expected.

Demo for restart: always

Add the following to your docker-compose.yml using

nano docker-compose.ymlversion: "3.7"services:alpine:image: alpine:3.8command: sleep 1restart: always

Bring up new container.

docker-compose up -d -t 0

Check its status:

docker ps -a

The sleep 1 causes the container to exit after just one second.

If you run this command repeatedly you will see the container continually restarting.

docker container stop your-container-id does not stop it.docker-compose down removes the container within seconds.

If you use docker-compose up to start up a container, use docker-compose down to take it down.

restart: always restarts containers that exit with zero ( success ) exit code.

If you replace command with:

command: sleep 1; exit 1

you can test that restart: always restarts containers that exit with non-zero ( failed ) exit code.

Demo for restart: on-failure

You learn new software by reading the docs and then using it.

Reading means zero experience.

Experiments means experience.

So based on what you experienced above, design and execute tests for restart: on-failure

It must test container restart on failure ( obviously )

It must also test if it will restart on successful exits ( exit return code 0 ).

Based on the above texts this should take 5 minutes maximum.

init: not set to true

From https://docs.docker.com/compose/compose-file/#init

Having this option set to true is important if you care about having processes inside your container shut down cleanly.

If you run docker container stop the processes inside the container must receive the correct signals forwarded to them. Then the shutdown subroutines for your applications are correctly signaled to shutdown cleanly.

If you have several programs or threads running in your container they must receive the STOP signal correctly managed by some overall manager process: INIT is that process.

You will see the explanation below demonstrated in real life example:

Init first sends SIGTERM ( signal 15 ) to let processes catch that signal to execute their clean shutdown routines. SIGTERM means — you must shut down, but do you clean shutdown routines first.

stop_grace_period = 10 seconds default.

stop_grace_period then waits 10 seconds for the container to exit CLEANLY before sending SIGKILL ( signal 9). SIGKILL immediately kills the proces — files not closed properly — no shutdown routines done. SIGKILL means process is running one moment; brains blown out the next.

Only with init: true does this critical important signals propagate properly through your container.

Summary : specify init: true if you care about preventing data corruption.

Important: This option got added in version 3.7 file format. The first line of your docker-compose.yml must read: version: “3.7”

First we will see how no init setting causes STOP to be handled badly.

Then we do it correctly by specifying init: true and observing the difference.

Add the following to your docker-compose.yml using

nano docker-compose.ymlversion: "3.7"services:alpine:image: alpine:3.8

Bring up new container ( no init anywhere ).

docker-compose up -d -t 0

Check its status and get container id:

docker ps -adocker exec -it  your-container-id  /bin/sh

Enter it and run ps

/ # psPID  USER  TIME  COMMAND1 root  0:00 sleep 60011 root  0:00 /bin/sh16 root  0:00 ps/ # exit

Note PID process 1 is not init.

Open a new shell console and run docker events

Back to original shell, if you now run

docker container stop your-container-id

you will see it takes 10 seconds to stop.

IMPORTANT: Look at the console output for docker events

Expected output :

2018-11-12T09:51:46.112995194+02:00 container kill b676e3bf51f9f1f99edc531ffa04b0ab164834cb218fe9b78abb65b396782d6a (com.docker.compose.config-hash=c19f3ca3bb22826fee98596d3cb40c4f403f8a51cede518945f5ef37bb989589, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, image=alpine:3.8, name=compose-tuts_alpine_1, signal=15)2018-11-12T09:51:56.134707392+02:00 container kill b676e3bf51f9f1f99edc531ffa04b0ab164834cb218fe9b78abb65b396782d6a (com.docker.compose.config-hash=c19f3ca3bb22826fee98596d3cb40c4f403f8a51cede518945f5ef37bb989589, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, image=alpine:3.8, name=compose-tuts_alpine_1, signal=9)2018-11-12T09:51:56.307015745+02:00 container die b676e3bf51f9f1f99edc531ffa04b0ab164834cb218fe9b78abb65b396782d6a (com.docker.compose.config-hash=c19f3ca3bb22826fee98596d3cb40c4f403f8a51cede518945f5ef37bb989589, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, exitCode=137, image=alpine:3.8, name=compose-tuts_alpine_1)2018-11-12T09:51:56.387969293+02:00 network disconnect 9802e2506ea80e7597f0b018450f86fa0c7dcc045a920702cc0e463aacfda84f (container=b676e3bf51f9f1f99edc531ffa04b0ab164834cb218fe9b78abb65b396782d6a, name=compose-tuts_default, type=bridge)2018-11-12T09:51:56.454351535+02:00 container stop b676e3bf51f9f1f99edc531ffa04b0ab164834cb218fe9b78abb65b396782d6a (com.docker.compose.config-hash=c19f3ca3bb22826fee98596d3cb40c4f403f8a51cede518945f5ef37bb989589, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, image=alpine:3.8, name=compose-tuts_alpine_1)

First event sends container SIGTERM — signal 15 text right at end of that line.

Docker determines container still does not shut down properly. Container ignores signal 15.

Second line: KILL signal event happens 10 seconds later — signal 9 text right at end of that line.

Docker kills container with signal 9 = KILL.

Summary: no init, no orderly shutdown, KILL had to be used.

init: true

Now observe how init handles the stop properly:

Add the following to your docker-compose.yml using

nano docker-compose.ymlversion: "3.7"services:alpine:image: alpine:3.8command: sleep 600init: true

Bring up new container with init: true

docker-compose up -d -t 0

Check its status and get container id:

docker ps -adocker exec -it  your-container-id  /bin/sh

Enter it and run ps

/ # psPID  USER  TIME  COMMAND1 root  0:00 /dev/init -- sleep 6006 root  0:00 sleep 6007 root  0:00 /bin/sh12 root  0:00 ps/ # exit

Note PID 1 runs /dev/init. Init is running PID 1: the only perfect place to manage sending signals to all processes in the container.

Press CTRL-c in the docker events console so that list of events are interrupted.

In this console run docker events again — now we going to observe INIT-managed shutdown events.

Back to original shell, do docker ps -a to get your container id.

If you now run

docker container stop your-container-id

you will see it stops very quickly.

IMPORTANT: Look at the console output for docker events

Expected output :

2018-11-12T09:50:14.892684432+02:00 container kill 7973ccec4849cf8759821d613e1b4870c68d76f396fefcfb1cd1bf1f7a3bd509 (com.docker.compose.config-hash=8d5184879de73d9f624319983822eafca40959cbe9f91929ff4648ace18acc6a, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, image=alpine:3.8, name=compose-tuts_alpine_1, signal=15)2018-11-12T09:50:15.019039627+02:00 container die 7973ccec4849cf8759821d613e1b4870c68d76f396fefcfb1cd1bf1f7a3bd509 (com.docker.compose.config-hash=8d5184879de73d9f624319983822eafca40959cbe9f91929ff4648ace18acc6a, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, exitCode=143, image=alpine:3.8, name=compose-tuts_alpine_1)2018-11-12T09:50:15.104203647+02:00 network disconnect 9802e2506ea80e7597f0b018450f86fa0c7dcc045a920702cc0e463aacfda84f (container=7973ccec4849cf8759821d613e1b4870c68d76f396fefcfb1cd1bf1f7a3bd509, name=compose-tuts_default, type=bridge)2018-11-12T09:50:15.149018229+02:00 container stop 7973ccec4849cf8759821d613e1b4870c68d76f396fefcfb1cd1bf1f7a3bd509 (com.docker.compose.config-hash=8d5184879de73d9f624319983822eafca40959cbe9f91929ff4648ace18acc6a, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, image=alpine:3.8, name=compose-tuts_alpine_1)

First event sends container SIGTERM — signal 15 text right at end of that line.

Docker determines container does shut down properly. Container properly processes signal 15. ( Container processes caught signal 15 and called their clean shutdown routines. )

The container die events happens 100 milliseconds later.

NO signal 9 = KILL sent.

Summary: init, fast orderly shutdown, no KILL used.

In short, I strongly suggesting using init: true. It’s easy, fast and saves corrupted data hassles.

Reference:https://www.alibabacloud.com/blog/practical-exercises-for-docker-compose-part-3_594415?spm=a2c41.12532335.0.0

Written by

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store