Docker Container Resource Management: CPU, RAM and IO: Part 3

— cpuset-cpus Using Sysbench

Syntax:

docker run -it --rm --name mybench  --cpuset-cpus 0,1 centos:bench /bin/sh -c 'time sysbench --threads=2 --events=4 --cpu-max-prime=800500 --verbosity=0 cpu run'real    0m1.957s
user 0m3.813s
sys 0m0.025s
docker run -it --rm --name mybench  --cpuset-cpus 0 centos:bench /bin/sh -c 'time sysbench --threads=2 --events=4 --cpu-max-prime=800500 --verbosity=0 cpu run'real    0m3.789s
user 0m3.740s
sys 0m0.029s
docker run -it --rm --name mybench  --cpuset-cpus 1 centos:bench /bin/sh -c 'time sysbench --threads=2 --events=4 --cpu-max-prime=800500 --verbosity=0 cpu run'real    0m3.809s
user 0m3.761s
sys 0m0.025s
docker run -it --rm --name mybench  --cpuset-cpus 0,1 centos:bench /bin/sh -c 'time sysbench --threads=2 --events=4 --cpu-max-prime=8005 --verbosity=0 cpu run'real    0m0.053s
user 0m0.023s
sys 0m0.017s
docker run -it --rm --name mybench  --cpuset-cpus 0 centos:bench /bin/sh -c 'time sysbench --threads=2 --events=4 --cpu-max-prime=8005 --verbosity=0 cpu run' real    0m0.066s
user 0m0.025s
sys 0m0.023s

— cpu-shares Using Sysbench

Earlier we determined this runs for .277 seconds using 2 threads.

for var in `seq 1 20`; do docker stats --no-stream ; done
docker run -d --name mybench100  --cpus 1 --cpu-shares=100  centos:bench /bin/sh -c 'for var in `seq 1 10`; do time sysbench --threads=1 --events=4 --cpu-max-prime=50500 --verbosity=0 cpu run ; done'
docker run -d --name mybench500 --cpus 1 --cpu-shares=500 centos:bench /bin/sh -c 'for var in `seq 1 10`; do time sysbench --threads=1 --events=4 --cpu-max-prime=50500 --verbosity=0 cpu run ; done'
docker run -d --name mybench1024 --cpus 1 --cpu-shares=1024 centos:bench /bin/sh -c 'for var in `seq 1 10`; do time sysbench --threads=1 --events=4 --cpu-max-prime=50500 --verbosity=0 cpu run ; done'
for var in `seq 1 20`; do docker stats --no-stream ; done
docker run -d --name mybench100  --cpu-shares=100  centos:bench /bin/sh -c 'sleep 4; for var in `seq 1 3`; do time sysbench --threads=1 --events=4 --cpu-max-prime=500500 --verbosity=0 cpu run ; done'
docker run -d --name mybench200 --cpu-shares=200 centos:bench /bin/sh -c 'sleep 3; for var in `seq 1 3`; do time sysbench --threads=1 --events=4 --cpu-max-prime=500500 --verbosity=0 cpu run ; done'
docker run -d --name mybench300 --cpu-shares=300 centos:bench /bin/sh -c 'sleep 2; for var in `seq 1 3`; do time sysbench --threads=1 --events=4 --cpu-max-prime=500500 --verbosity=0 cpu run ; done'
docker run -d --name mybench600 --cpu-shares=600 centos:bench /bin/sh -c 'for var in `seq 1 4`; do time sysbench --threads=1 --events=4 --cpu-max-prime=500500 --verbosity=0 cpu run ; done'
CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
d3744f5a52ee mybench600 94.14% 1.23MiB / 985.2MiB 0.12% 508B / 0B 13.1MB / 0B 0
5c694381ef2f mybench300 50.96% 1.34MiB / 985.2MiB 0.14% 508B / 0B 12.4MB / 0B 0
8894176a9b72 mybench200 34.10% 1.215MiB / 985.2MiB 0.12% 508B / 0B 12.4MB / 0B 0
8b783befba46 mybench100 15.71% 1.223MiB / 985.2MiB 0.12% 578B / 0B 12.4MB / 0B 0
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
d3744f5a52ee mybench600 93.73% 1.348MiB / 985.2MiB 0.14% 578B / 0B 13.1MB / 0B 0
5c694381ef2f mybench300 55.25% 1.219MiB / 985.2MiB 0.12% 578B / 0B 13.1MB / 0B 0
8894176a9b72 mybench200 33.25% 1.223MiB / 985.2MiB 0.12% 578B / 0B 12.4MB / 0B 0
8b783befba46 mybench100 16.21% 1.227MiB / 985.2MiB 0.12%
for var in `seq 1 20`; do docker stats --no-stream ; done
docker run -d --name mybench100 --cpuset-cpus 0 --cpu-shares=100  centos:bench /bin/sh -c 'sleep 4; for var in `seq 1 3`; do time sysbench --threads=1 --events=4 --cpu-max-prime=500500 --verbosity=0 cpu run ; done'
docker run -d --name mybench200 --cpuset-cpus 0 --cpu-shares=200 centos:bench /bin/sh -c 'sleep 3; for var in `seq 1 3`; do time sysbench --threads=1 --events=4 --cpu-max-prime=500500 --verbosity=0 cpu run ; done'
docker run -d --name mybench300 --cpuset-cpus 0 --cpu-shares=300 centos:bench /bin/sh -c 'sleep 2; for var in `seq 1 3`; do time sysbench --threads=1 --events=4 --cpu-max-prime=500500 --verbosity=0 cpu run ; done'
docker run -d --name mybench600 --cpuset-cpus 0 --cpu-shares=600 centos:bench /bin/sh -c 'for var in `seq 1 4`; do time sysbench --threads=1 --events=4 --cpu-max-prime=500500 --verbosity=0 cpu run ; done'
CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
61a4f3d31614 mybench600 49.88% 1.215MiB / 985.2MiB 0.12% 648B / 0B 13.1MB / 0B 0
199ee60e6e1d mybench300 24.92% 1.348MiB / 985.2MiB 0.14% 648B / 0B 13.1MB / 0B 0
ee9b719a6d88 mybench200 16.77% 1.23MiB / 985.2MiB 0.12% 648B / 0B 12.4MB / 0B 0
016fc5d86d68 mybench100 8.26% 1.227MiB / 985.2MiB 0.12% 648B / 0B 12.4MB / 0B 0
docker run -d --name mybench100  --cpus .1 --cpu-shares=100  centos:bench /bin/sh -c 'sleep 2; for var in `seq 1 4`; do time sysbench --threads=1 --events=4 --cpu-max-prime=500500 --verbosity=0 cpu run ; done'
docker run -d --name mybench200 --cpus .1 --cpu-shares=200 centos:bench /bin/sh -c 'sleep 2; for var in `seq 1 3`; do time sysbench --threads=1 --events=4 --cpu-max-prime=500500 --verbosity=0 cpu run ; done'
docker run -d --name mybench300 --cpus .1 --cpu-shares=300 centos:bench /bin/sh -c 'sleep 1; for var in `seq 1 3`; do time sysbench --threads=1 --events=4 --cpu-max-prime=500500 --verbosity=0 cpu run ; done'
docker run -d --name mybench600 --cpus .1 --cpu-shares=600 centos:bench /bin/sh -c 'for var in `seq 1 3`; do time sysbench --threads=1 --events=4 --cpu-max-prime=500500 --verbosity=0 cpu run ; done'
docker stats --no-stream outputCONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
d1efffa6b398 mybench600 9.98% 1.223MiB / 985.2MiB 0.12% 648B / 0B 13.1MB / 0B 0
e5761b1fd9ae mybench300 9.94% 1.227MiB / 985.2MiB 0.12% 648B / 0B 13.1MB / 0B 0
d25746948b3d mybench200 9.95% 1.348MiB / 985.2MiB 0.14% 648B / 0B 13.1MB / 0B 0
9003482281bd mybench100 9.94% 1.227MiB / 985.2MiB 0.12%

— cpuset-mems

Only works on servers with NUMA architecture, for example Opterons and Nelahem Xeons.

  • — cpuset-cpus are used for non-NUMA servers
  • — cpuset-mems are used for NUMA servers

— memory and — memory-swap (No Swapping Allowed) Using Stress Bench Tool

Run our container, limiting RAM: — memory=20m — memory-swap=20m

  • Allocates 1 vm worker
  • Allocates 18 MB of RAM
sh-4.2# stress --vm 1 --vm-bytes 18M
stress: info: [50] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
sh-4.2# stress --vm 1 --vm-bytes 19M
stress: info: [53] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
stress: FAIL: [53](415) <-- worker 54 got signal 9
stress: WARN: [53](417) now reaping child worker processes
stress: FAIL: [53](451) failed run completed in 0s
sh-4.2# stress --vm 1 --vm-bytes 17M
stress: info: [64] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
sh-4.2# stress --vm 1 --vm-bytes 17M --vm-hang 3
stress: info: [70] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
sh-4.2# stress --vm 1 --vm-bytes 30M --vm-hang 3
stress: info: [72] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
stress: FAIL: [72](415) <-- worker 73 got signal 9
stress: WARN: [72](417) now reaping child worker processes
stress: FAIL: [72](451) failed run completed in 0s
sh-4.2# exit

— memory and — memory-swap (Swapping Allowed) Using Stress Bench Tool

Run our container, limiting RAM: — memory=20m — memory-swap=30m

sh-4.2# stress --vm 1 --vm-bytes 25M --vm-hang 3
stress: info: [9] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
Tasks: 125 total,   2 running, 123 sleeping,   0 stopped,   0 zombie
%Cpu(s): 0.3 us, 4.9 sy, 0.0 ni, 83.5 id, 11.1 wa, 0.0 hi, 0.2 si, 0.0 st
MiB Mem : 985.219 total, 460.227 free, 166.090 used, 358.902 buff/cache
MiB Swap: 1499.996 total, 1492.684 free, 7.312 used. 636.930 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND SWAP
2565 root 20 0 32.3m 18.0m S 9.0 1.8 0:24.97 stress 7.1m
sh-4.2# stress --vm 1 --vm-bytes 35M --vm-hang 3
stress: info: [11] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
stress: FAIL: [11](415) <-- worker 12 got signal 9
stress: WARN: [11](417) now reaping child worker processes
stress: FAIL: [11](451) failed run completed in 0s
sh-4.2# exit

— memory-swap Is Not Set

docker container run -ti --rm  --memory=20m --memory-swap=30m --name memtest centos:bench /bin/sh
sh-4.2# stress --vm 1 --vm-bytes 35M --vm-hang 3
stress: info: [9] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
^C
sh-4.2# stress --vm 1 --vm-bytes 38M --vm-hang 3
stress: info: [11] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
stress: FAIL: [11](415) <-- worker 12 got signal 9
stress: WARN: [11](417) now reaping child worker processes
stress: FAIL: [11](451) failed run completed in 1s
sh-4.2# stress --vm 1 --vm-bytes 37M --vm-hang 3
stress: info: [13] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
^C
sh-4.2# exit

Read/Write : IO Rate Limiting

These settings allow you to limit the IO rates of your containers:

  • — device-read-bps Limit read rate (bytes per second) from a device
  • — device-read-iops Limit read rate (IO per second) from a device
  • — device-write-bps Limit write rate (bytes per second) to a device
  • — device-write-iops Limit write rate (IO per second) to a device
  • — io-maxbandwidth Maximum IO bandwidth limit for the system drive (Windows only)
  • — io-maxiops Maximum IOps limit for the system drive (Windows only)
df -h
docker run -it --rm alpine:3.8 /bin/sh
df -h
sh-4.2# time dd if=/dev/zero of=test.out bs=1M count=10 oflag=direct
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 10.0027 s, 1.0 MB/s
real 0m10.009s
user 0m0.001s
sys 0m0.015s
sh-4.2# exit
exit

Compose File Resource Limits

This tutorial only gave you experience with CPU and RAM resource limits using the easy RUN command settings.

— cap-add=sys_nice

Syntax:

sh-4.2#  renice -n -10 1
renice: failed to set priority for 1 (process ID): Permission denied
docker run -it --rm --name mybench --cap-add=sys_nice centos:bench /bin/shsh-4.2# renice -n -10 1
1 (process ID) old priority 0, new priority -10
sh-4.2# renice -n -19 1
1 (process ID) old priority -10, new priority -19
sh-4.2# renice -n 19 1
1 (process ID) old priority -19, new priority 19
sh-4.2# exit
exit

Conclusion

In this tutorial series, we’ve explored and experimented with various Docker resource limitation commands. You should use actual Linux benchmarking tools as sysbench and stress as these tools were specifically designed to give you repeatable results.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alibaba Cloud

Alibaba Cloud

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website:https://www.alibabacloud.com