Exploring Alibaba Group’s PouchContainer Resource Management APIs — Part 2

1. Memory Resource Management

1.1 -m, — memory

It can limit the amount of memory used by the container, and the corresponding cgroup file is cgroup/memory/memory.limit_in_bytes.

# pouch run -ti --memory 100M reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash -c "cat /sys/fs/cgroup/memory/memory.limit_in_bytes"
104857600
# free -m
total used free shared buff/cache available
Mem: 257755 2557 254234 1 963 254903
Swap: 2047 0 2047
# pouch run -ti --memory 100M reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04stress stress --vm 1 --vm-bytes 150M
stress: info: [1] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
#free -m
total used free shared buff/cache available
Mem: 257755 2676 254114 1 965 254783
Swap: 2047 41 2006
# pouch run -ti --memory 100M reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04stress stress --vm 1 --vm-bytes 150M
stress: info: [1] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
stress: FAIL: [1](422) kill error: No such process
stress: FAIL: [1](452) failed run completed in 0s
esses
stress: FAIL: [1](422) kill error: No such process
stress: FAIL: [1](452) failed run completed in 0s

1.2 — memory-swap

It can limit the total amount of swap partition and memory used by the container, and the corresponding cgroup file is cgroup/memory/memory.memsw.limit_in_bytes.

# pouch run -ti -m 300M --memory-swap 1G reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash -c "cat /sys/fs/cgroup/memory/memory.memsw.limit_in_bytes"
1073741824
# pouch run -ti -m 100M --memory-swap 200M reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04stress bash -c "stress --vm 1 --vm-bytes 300M"
stress: info: [1] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
stress: FAIL: [1](416) <-- worker 10 got signal 9
stress: WARN: [1](418) now reaping child worker processes
stress: FAIL: [1](422) kill error: No such process
stress: FAIL: [1](452) failed run completed in 0s

1.3 — memory-swappiness

This API sets the trend for the container to use the swap partition, which is an integer ranging from 0 to 100 (inclusive). 0 indicates that the container does not use the swap partition, and 100 indicates that the container uses the swap partition as much as possible. The corresponding cgroup file is cgroup/memory/memory.swappiness.

# pouch run -ti --memory-swappiness=100 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash -c 'cat /sys/fs/cgroup/memory/memory.swappiness'
100

1.4 — memory-wmark-ratio

Used to calculate low_wmark, low_wmark = memory.limit_in_bytes * MemoryWmarkRatio. When memory.usage_in_bytes is greater than low_wmark, the kernel thread is triggered to perform memory reclamation. When the memory.usage_in_bytes is less than high_wmark, the reclamation is stopped. The corresponding cgroup file is cgroup/memory/memory.wmark_ratio.

# pouch run -ti --memory-wmark-ratio=60 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash -c 'cat /sys/fs/cgroup/memory/memory.wmark_ratio'
60

1.5 — oom-kill-disable

When out-of-memory (OOM) occurs, the system kills the container process by default. If you do not want the container process to be killed, you can use this API. This API corresponds to the cgroup file cgroup/memory/memory.oom_control.

# pouch run -m 20m --oom-kill-disable=true reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash -c 'cat /sys/fs/cgroup/memory/memory.oom_control'
oom_kill_disable 1
under_oom 0
# pouch run -m 20m --oom-kill-disable=false reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash -c 'x=a; while true; do x=$x$x$x$x; done'[root@r10d08216.sqa.zmf /root]
#echo $?
137

1.6 — oom-score-adj

The parameter — oom-score-adj sets the possibility that the container process will trigger OOM. The larger the value, the easier the OOM of the container process will be triggered. When the value is -1000, the container process does not trigger OOM at all. This option corresponds to the underlying API /proc/$pid/oom_score_adj.

# pouch run -ti --oom-score-adj=300 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04stress bash -c "cat /proc/self/oom_score_adj"
300

2. CPU resource management

2.1 — cpu-period

The cycle of the kernel default Linux CFS (Completely Fair Scheduler) is 100 ms; we use — cpu-period to set the CPU usage cycle for the container, and the API — cpu-period needs to be used with the API — cpu-quota. The API — cpu-quota sets the value of CPU usage. CFS is the default scheduling model used by the kernel to allocate CPU resources to running processes. For multi-core CPUs, the value of — cpu-quota is adjusted as needed.

# pouch run -ti --cpu-period 50000 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash -c "cat /sys/fs/cgroup/cpu/cpu.cfs_period_us"
50000
# pouch run -ti --cpu-period=50000 --cpu-quota=25000 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04stress stress -c 1
stress: info: [1] dispatching hogs: 1 cpu, 0 io, 0 vm, 0 hdd
# top -n1
top - 17:22:40 up 1 day, 57 min, 3 users, load average: 0.68, 0.16, 0.05
Tasks: 431 total, 2 running, 429 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.0 us, 0.1 sy, 0.0 ni, 99.9 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 26354243+total, 25960588+free, 1697108 used, 2239424 buff/cache
KiB Swap: 2096636 total, 0 free, 2096636 used. 25957392+avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
53256 root 20 0 7324 100 0 R 50.0 0.0 0:12.95 stress

2.2 — cpu-quota

It corresponds to the cgroup file cgroup/cpu/cpu.cfs_quota_us.

# pouch run -ti --cpu-quota 1600 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash -c "cat /sys/fs/cgroup/cpu/cpu.cfs_quota_us"
1600

2.3 — cpu-share

It corresponds to the cgroup file cgroup/cpu/cpu.shares.

# pouch run -ti --cpu-quota 1600 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash -c "cat /sys/fs/cgroup/cpu/cpu.shares"
1600
# pouch run -d --cpuset-cpus=0 --cpu-share 1024 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04stress stress -c 1
c7b99f3bc4cf1af94da35025c66913d4b42fa763e7a0905fc72dce66c359c258
[root@r10d08216.sqa.zmf /root]
# pouch run -d --cpuset-cpus=0 --cpu-share 512 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04stress stress -c 1
1ade73df0dd9939cc65e05117e3b0950b78079fb36f6cc548eff8b20e8f5ecb9
#top
top - 09:38:24 up 3 min, 2 users, load average: 1.20, 0.34, 0.12
Tasks: 447 total, 3 running, 444 sleeping, 0 stopped, 0 zombie
%Cpu(s): 3.1 us, 0.0 sy, 0.0 ni, 96.9 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 26354243+total, 26187224+free, 964068 used, 706120 buff/cache
KiB Swap: 2096636 total, 2096636 free, 0 used. 26052548+avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10513 root 20 0 7324 100 0 R 65.1 0.0 0:48.22 stress
10687 root 20 0 7324 96 0 R 34.9 0.0 0:20.32 stress

2.4 — cpuset-cpus

This API corresponds to the cgroup file cgroup/cpuset/cpuset.cpus.

# pouch run -ti --cpuset-cpus 1 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash -c "cat /sys/fs/cgroup/cpuset/cpuset.cpus"
1
# pouch run -ti --cpuset-cpus 1 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 stress -c 1
#top
top - 17:58:38 up 1 day, 1:33, 3 users, load average: 0.51, 0.11, 0.04
Tasks: 427 total, 2 running, 425 sleeping, 0 stopped, 0 zombie
%Cpu0 : 0.0 us, 0.3 sy, 0.0 ni, 99.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu1 :100.0 us, 0.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu2 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu3 : 0.0 us, 0.3 sy, 0.0 ni, 99.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st

2.5 — cpuset-mems

This API corresponds to the cgroup file cgroup/cpuset/cpuset.mems.

# pouch run -ti --cpuset-mems=0 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash -c "cat /sys/fs/cgroup/cpuset/cpuset.mems"
0
# pouch run -ti --cpuset-mems="1,3" reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash
# pouch run -ti --cpuset-mems="0-2" reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash

3. IO resource management

3.1 — blkio-weight

The container’s block device IO weight can be set by the API — blkio-weight, which is an integer ranging from 10 to 1,000 (inclusive). By default, all containers get the same weight value (500). It corresponds to the cgroup file cgroup/blkio/blkio.weight. The following command sets the IO weight of the container block device to 10, and you can see that the value of the corresponding cgroup file is 10 in the log.

# pouch run -ti --rm --blkio-weight 10 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash -c "cat /sys/fs/cgroup/blkio/blkio.weight"
10
# pouch run -it --name c1 --blkio-weight 300 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 /bin/bash
# pouch run -it --name c2 --blkio-weight 600 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 /bin/bash
# time dd if=/mnt/zerofile of=test.out bs=1M count=1024 oflag=direct

3.2 — blkio-weight-device

The container’s specific block device IO weight can be set by the API — blkio-weight-device=”deviceName:weight”, which is an integer ranging from 10 to 1,000 (inclusive).

# pouch run --blkio-weight-device "/dev/sda:1000" reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash -c "cat /sys/fs/cgroup/blkio/blkio.weight_device"
8:0 1000
#stat -c %t:%T /dev/sda
8:0
# pouch run --blkio-weight 300 --blkio-weight-device "/dev/sda:500" reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash -c "cat /sys/fs/cgroup/blkio/blkio.weight_device"
8:0 500

3.3 — device-read-bps

This API limits the read rate of a specified device. The unit can be KB, MB, or GB. It corresponds to the cgroup file cgroup/blkio/blkio.throttle.read_bps_device.

# pouch run -it --device /dev/sda:/dev/sda --device-read-bps /dev/sda:1mb reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash -c "cat /sys/fs/cgroup/blkio/blkio.throttle.read_bps_device"
8:0 1048576
# pouch run -it --device /dev/sda:/dev/sda --device-read-bps /dev/sda:500k reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash
root@r10f10195:/# dd iflag=direct,nonblock if=/dev/sda of=/dev/null bs=5000k coun
1+0 records in
1+0 records out
5120000 bytes (5.1 MB) copied, 10.2738 s, 498 kB/s

3.4 — device-write-bps

This API limits the write rate of a specified device. The unit can be KB, MB, or GB. It corresponds to the cgroup file cgroup/blkio/blkio.throttle.write_bps_device.

# pouch run -it --device /dev/sda:/dev/sda --device-write-bps /dev/sda:1mB reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash -c "cat /sys/fs/cgroup/blkio/blkio.throttle.write_bps_device"
8:0 1048576
# pouch run -it --device /dev/sdb:/dev/sdb --device-write-bps /dev/sdb:1mB reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash
root@r10d08216:/# dd oflag=direct,nonblock of=/dev/sdb if=/dev/urandom bs=10K count=1000
1024+0 records in
1024+0 records out
10485760 bytes (10 MB) copied, 10.0022 s, 1.0 MB/s

3.5 — device-read-iops

This API sets the IO read rate of the device, and corresponds to the cgroup file cgroup/blkio/blkio.throttle.read_iops_device.

# pouch run -it --device /dev/sda:/dev/sda --device-read-iops /dev/sda:400 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash -c "cat /sys/fs/cgroup/blkio/blkio.throttle.read_iops_device"
8:0 400
# pouch run -it --device /dev/sda:/dev/sda --device-read-iops /dev/sda:400 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash
root@r10d08216:/# dd iflag=direct,nonblock if=/dev/sda of=/dev/null bs=1k count=1024
1024+0 records in
1024+0 records out
1048576 bytes (1.0 MB) copied, 2.51044 s, 418 kB/s
root@r10d08216:/#

3.6 — device-write-iops

This API sets the IO write rate of the device, and corresponds to the cgroup file cgroup/blkio/blkio.throttle.write_iops_device.

# pouch run -it --device /dev/sda:/dev/sda --device-write-iops /dev/sda:400 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash -c "cat /sys/fs/cgroup/blkio/blkio.throttle.write_iops_device"
8:0 400
# pouch run -it --device /dev/sdb:/dev/sdb --device-write-iops /dev/sdb:400 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04 bash
root@r10d08216:/# dd oflag=direct,nonblock of=/dev/sdb if=/dev/urandom bs=1K count=1024
1024+0 records in
1024+0 records out
1048576 bytes (1.0 MB) copied, 2.50754 s, 418 kB/s

3.7 Other resource management APIs

— pids-limit

# pouch run -ti --pids-limit 100 reg.docker.alibaba-inc.com/sunyuan/ubuntu:14.04stress bash -c "cat /sys/fs/cgroup/pids/pids.max"
100
bash: fork: retry: Resource temporarily unavailable
bash: fork: retry: Resource temporarily unavailable

4. Summary

PouchContainer’s resource management depends on the underlying technology of the Linux kernel. You can add some targeted tests as needed to learn more. The implementation of the kernel technology on which it depends is far beyond the scope of this article. You can read the kernel manual for more information: PouchContainer community document.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alibaba Cloud

Alibaba Cloud

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website:https://www.alibabacloud.com