Docker Container Cleanup Tutorial

By Alwyn Botha, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud’s incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.

This tutorial gives you practical experience in cleaning up Docker containers.

You need only about a week’s experience with Docker to be able to understand most of this tutorial.

You need more experience as Linux administrator, since grep, awk and xargs is also used.

grep, awk and xargs are very powerful tools to use at the shell command line. It is worthwhile spending a day reading and executing tutorials on these tools. This tutorial focus on Docker; it will only give ten line descriptions where grep, awk and xargs are used. ( It only explains the one percent of grep, awk and xargs functionality we need to understand in the context of using it in docker cleanups. )

Preparation

To follow this tutorial you need an Alibaba Cloud Elastic Compute Service (ECS) server with Docker already installed. If you are not sure how, you can refer to this tutorial.

In order to learn about cleaning up containers we need some containers to clean up.

First we need some images from https://hub.docker.com/

For the purposes of this tutorial it does not matter at all what is inside those images. You need zero experience of Nginx, Redis and Alpine to understand this full tutorial completely.

Redis is an open source, in-memory data structure store, used as a database and data cache. Most programming languages can use it to make the programs faster via near-instant access to data in memory.

NGINX is a free, open-source, high-performance HTTP server: competing directly with more well-known Apache web server.

Alpine Linux is a 5MB Linux distribution, very popular with docker users.

Alpine Linux is used as a base image for docker applications. It provides a small Linux environment for running apps.

Enter the following at your shell prompt to get these 4 images downloaded onto your computer.

The docker pull command downloads Docker images from a registry. A docker registry contains a collection of Docker images.

By default, the docker pull command pulls / downloads docker images from Docker Hub: https://hub.docker.com/

Docker images are used to create running containers. Docker images contain all the software needed to run a specific application. For example, the nginx:mainline-alpine image you downloaded contains all the software needed to run Nginx in a container.

In docker pull nginx:mainline-alpine the nginx:mainline-alpine is the name of the image we want to pull / download. The image name is nginx.

The mainline-alpine is a tag. Tags are used to differentiate different versions of the same image, for example: there could be nginx:centos, nginx:ubuntu, nginx:debian8 and nginx: debian9 images at the Docker Hub repository. We use tags to specify exactly which image to download.

In the case of docker pull alpine:3.8 the 3.8 tag represents a version number.

Run:

Expected output:

If you are running this on a fresh docker install server you will have no running containers.

Let’s make a mess and create several containers to clean up. Enter these commands:

The docker run will turn the static images you downloaded before into executing containers.

A container is a runtime version of a docker image. The container contains all the software from the Docker image as well as a complete execution environment including running processes.

We do not need nginx or redis functionality in this tutorial. We only need an interesting list of varied containers. Then we can better learn how to selectively stop and delete containers.

Let’s stop nginx and redis. The hello-world container only displays a message and then exits. There is no need to stop it, its already stopped.

List all containers by entering this command:

Expected output

We now have 5 stopped containers

We cannot use

That command will prune/remove ALL stopped containers. Let’s assume we still want to look at nginx and redis logs. Therefore we must carefully prune just the hello-world containers.

Let’s build a list of JUST the hello world containers. Run

grep will select only the output lines that contain the text: hello-world

grep is a Linux shell utility ava on most Linux distributions. It was written 40 years ago.

Expected output

Perfect. We need a list of just the containers we want to prune.

Alternatively we could run:

The — invert-match inverts the matching, it only selects non-matching lines. -v is shortcut form for — invert-match

“redis|nginx” is a regular expression that selects only text containing redis or nginx. Here the pipe symbol | means or.

Here egrep will select only the output lines that EXCLUDES the text redis and EXCLUDES the text nginx

We use egrep here: this grep supports extended regular expressions: “redis|nginx” is the extended regular expression used in this case.

The format of docker container rm is:

We need just a list of container IDs to pass to docker container rm

Container IDs are field number one of the output. awk ‘{print $1}’ prints just the first field passed to it.

AWK is a programming language designed for text processing . It is used to extract and filter text data. It is a installed on most Linux distributions.

AWK was created in 1970. The AWK name is derived from the surnames of its authors桝lfred Aho, Peter Weinberger, and Brian Kernighan.

awk and grep both filters text. They each have unique features. grep cannot print input columns selectively. It is a feature only awk has. So we have to use grep and awk.

awk ‘{print $1}’ prints just the first field passed to it. awk by default assumes fields are separated by white space characters. Using __ awk ‘{print $1}’ __ in our case will print the containers IDs, the first field / column in docker ps -a output.

Run:

Expected output

Let’s remove all 3 those hello-world containers. Pick one of the following commands to run:

or

xargs docker container rm receives the container IDs one by one and removes them.

Expected output

Perfect. docker container rm shows the container IDs that it removed.

Run:

redis and nginx still exists.

Let’s investigate those redis and nginx logs so that we can remove those containers as well. docker logs shows the logs of a container.

Run:

There is nothing shown. We did not even use nginx container for anything.

Mini investigation of logs complete. Now we can remove all containers.

Run:

2 container IDs shown. Those containers are now deleted as well.

Run:

Expected output

As expected, no containers running or stopped.

Summary

First we learnt to remove containers by grepping only the text we want then selecting just the container IDs via awk.

Second we pass the container IDs to xargs docker container rm which removes the passed containers.

alternatively:

Remove Containers Using Docker Container Prune

Run it twice more so that we have 3 containers to cleanup.

Run:

It shows nothing since it only shows running containers.

Run:

The -a show all containers; including stopped ones.

Expected output

The command docker container prune removes all stopped containers

The command docker container prune — force removes all stopped containers but do not prompt for confirmation

Let’s run

Expected output

Total reclaimed space: 0B

3 containers deleted. On prune option the -f is shortcut for — force.

Prune containers using labels

Labels are used in docker to group related containers. Containers can then be selectively processed by selecting it based on label values.

The command docker container prune supports deleting containers using labels as filters.

The currently supported filters are:

until () — only remove containers created before given timestamp

label — only remove containers with (or without, in case label!=… is used) the specified labels.

Let’s create 3 containers using labels this time.

Execute these 3 commands:

The docker ps -a command does not show labels. To show labels we need to use the — format option.

Complete — format reference at

https://docs.docker.com/engine/reference/commandline/ps/#formatting

Run:

Expected output

Let’s selectively prune just the hello-1 container by running:

Expected output

Rerun the command:

Now only 2 containers remain.

Let’s prune these 2 containers by running:

then run

Expected output

All containers pruned. Labels are a good way to selectively prune one container with a specific label.

It just as useful to prune several containers if they all share the same label.

Docker ps Filtering Reference

From https://docs.docker.com/engine/reference/commandline/ps/#filtering

name::: Container抯 name

label::: An arbitrary string representing either a key or a key-value pair. Expressed as or =

ancestor::: Filters containers which share a given image as an ancestor. Expressed as [:], , or

volume::: Filters running containers which have mounted a given volume or bind mount.

network::: Filters running containers connected to a given network.

publish or expose::: Filters containers which publish or expose a given port. Expressed as port[/protocol]

Since these 3 containers share the same ancestor image: __ hello-word:latest__, they could easily have been pruned with this one command:

If these 3 containers used the same network for example: __ mytestnetwork__, they could easily have been pruned with this one command:

Prune Containers Using Exit Status

Exit status represent the success status of the processes in an exited container.

Exit status of zero normally means success ( across operating systems and programming languages: exit status 0 = success ).

The specific numeric value of the exit status has predefined value for specific programming languages and programs: for example: 404 means webpage not found in Nginx and Apache web server.

The 123 and 9 exit status below are just random numbers, used to demonstrate how to clean up containers with different exit statuses.

Run the following commands to create 5 containers having 4 different exit status codes:

Run the following to show the list of containers and their status exit codes:

Expected output

nginx and redis are running, they have no exit codes yet.

Now run:

Expected output

Only 3 exited containers shown since we specified — filter status=exited

Expected output

Only 1 exited containers shown: the one with exited status = 0

Run

Expected output

Only 1 exited containers shown: the one with exited status = 123

Let’s remove container with exit status = 0. Run:

Expected output

Run:

Expected output

exited=0 container removed success.

Why the error:

Error: No such container: CONTAINER

Run:

Expected output

That CONTAINER header is the problem. We should only pass a list of container IDs to xargs docker container rm.

To fix the problem remove that CONTAINER header line using grep: grep -v CONTAINER

-v is shortcut form for — invert-match.

Run:

Expected output

Success: no CONTAINER header line shown.

These 2 lines have identical results:

Here docker ps filters exited 9 status code containers, awk prints first field, grep selects lines EXCLUDING the words CONTAINER.

alternative:

Here docker ps filters exited 9 status code containers, grep selects lines EXCLUDING the words CONTAINER, awk prints first field.

In general I would do selecting via grep first and then as a final step use awk to print only the required columns. Logically the steps flow better: grep selecting, then final awk print results.

Let’s now remove the container with exited status 9 without getting the error message

Run:

Expected output

Success: container removed and no error shown.

Let’s now remove the container with exited status 123 without getting the error message

Run:

Expected output

Success: container removed and no error shown.

Run:

Expected output

Only nginx and redis still running.

Remove Containers in Status: Created

Sometimes you end up with a container in a created status. That happens when the container cannot get in the running status because of some error.

Let’s create a container in a created status by deliberately cause an error during its startup run phase.

Run

Expected output

Run:

Expected output

The zxgarbagez error causes the created status.

Let’s list just such created containers.

Expected output

Let’s remove all containers in status=created

Run

Expected output

Name of container shown as it gets removed.

Run:

Expected output

Only nginx and redis still running.

Remove Containers in Status: Paused

In this section we are going to pause a container and then attempt to remove it.

Docker uses the cgroups freezer functionality to freeze / pause all processes in a container.

With the cgroups freezer functionality the processes in the container are unaware, and unable to prevent or ignore the freeze command.

Run:

This starts up a container using the tiny alpine Linux distribution. It uses the /bin/sh command ( a small version of /bin/bash ). It runs the sleep command ( doing nothing ) for 10 minutes. This gives us a running container that we can pause.

Run:

Expected output

Run:

Expected output

Notice it is now in paused status.

If you run the command below you will get identical output list:

Let’s attempt to remove this container. Run:

Expected output

Please read the output. Let’s stop the container by running

Expected output

Shows testme got stopped.

When you have several paused containers you wish to stop, you can stop them all in one go using:

This will stop ALL containers in paused status.

Let’s investigate if that worked. Run

Expected output

Obviously there is no stopped status. When you stop a container it goes into the exited status.

List of valid container statuses: created, restarting, running, removing, paused, exited, or dead

Let’s investigate if that worked, this time using the correct status: exited. Run

Expected output

Success. This is our list of currently exited containers. Let’s pass this list of container IDs to docker container rm to remove these containers. Run

Expected output

Success. Our one exited container is now removed. You can run docker ps -a all by yourself and see this container no longer appears on the list.

Paused containers can be unpaused to start off at the exact place it got paused. A stopped container cannot be restarted at the exact place it got stopped. When you restart a container it starts at the beginning.

Summary: You cannot prune / remove a paused container — You have to unpause and then stop the container.

Remove the Container When It Exits

You already saw the hello world containers had to be manually removed / pruned.

If you run containers using the — rm option, the container is automatically removed when it exits

Let’s get practical experience with this. Then you will probably always use the convenient — rm option.

Run these 3 commands to create 3 hello world containers:

Let’s check the list of containers using

Expected output:

No exited hello world containers shown. The — rm automatically removed those 3 containers when they exited. No cleanup needed.

Define these bash aliases. You can then use the second one for quick tests, saving you cleanup afterwards.

alias dr=’docker run’

alias drrm=’docker run — rm’

A bash alias is a shorter version of long shell commands. Its purpose is to avoid typing a long command.

Tutorial Cleanup

This tutorial is now at its end.

Run these commands to remove all the images you will no longer be using.

Overall Docker Cleanup

Maybe your development docker server is a mess.

Maybe ALL your containers should be stopped.

You can quickly do that using:

It passes these IDs to docker stop.

ALL containers are stopped. No errors are shown for already exited containers.

To now remove all containers, run

Summary

Now you can experiment by applying container cleanup commands at your work.

Pro safety tip: always run a command to show you the selected list of containers to act upon. Only then run docker container rm to actually remove the containers.

grep, awk and xargs were used extensively in this tutorial. If you are not familiar with these tools you can now see it is really worthwhile to read some tutorials on it.

Reference:https://www.alibabacloud.com/blog/docker-container-cleanup-tutorial_594334?spm=a2c41.12475574.0.0

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store