Kubernetes Demystified: Restrictions on Java Application Resources

git clone https://github.com/denverdino/system-info
cd system-info`
  1. The app in the pod is an initialization container, responsible for copying one JSP application to the “webapps” directory of the Tomcat container. Note: In the image, the JSP application index.jsp is used to display JVM and system resource information.
  2. The Tomcat container remains active and we have restricted the maximum memory usage to 256 MB.
apiVersion: v1
kind: Pod
metadata:
name: test
spec:
initContainers:
- image: registry.cn-hangzhou.aliyuncs.com/denverdino/system-info
name: app
imagePullPolicy: IfNotPresent
command:
- "cp"
- "-r"
- "/system-info"
- "/app"
volumeMounts:
- mountPath: /app
name: app-volume
containers:
- image: tomcat:9-jre8
name: tomcat
imagePullPolicy: IfNotPresent
volumeMounts:
- mountPath: /usr/local/tomcat/webapps
name: app-volume
ports:
- containerPort: 8080
resources:
requests:
memory: "256Mi"
cpu: "500m"
limits:
memory: "256Mi"
cpu: "500m"
volumes:
- name: app-volume
emptyDir: {}
$ kubectl create -f test.yaml
pod "test" created
$ kubectl get pods test
NAME READY STATUS RESTARTS AGE
test 1/1 Running 0 28s
$ kubectl exec test curl http://localhost:8080/system-info/
...
$ kubectl exec test curl http://localhost:8080/system-info/ | html2textJava version     Oracle Corporation 1.8.0_162
Operating system Linux 4.9.64
Server Apache Tomcat/9.0.6
Memory Used 29 of 57 MB, Max 878 MB
Physica Memory 3951 MB
CPU Cores 2
**** Memory MXBean ****
Heap Memory Usage init = 65011712(63488K) used = 19873704(19407K) committed
= 65536000(64000K) max = 921174016(899584K)
Non-Heap Memory Usage init = 2555904(2496K) used = 32944912(32172K) committed =
33882112(33088K) max = -1(-1K)
  1. If we do not set a JVM heap size, the maximum heap size is set by default based on the memory size of the host environment.
  2. Docker containers use cgroups to limit the resources used by processes. Therefore, if the JVM in the container still uses the default settings based on the host environment memory and CPU cores, this results in incorrect JVM heap calculation.

Solutions

Enable cgroup resource awareness

The Java community was also aware of this problem and now supports auto sensing of container resource restrictions in Java SE 8u131+ and JDK 9: https://blogs.oracle.com/java-platform-group/java-se-support-for-docker-cpu-and-memory-limits

java -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap ⋯
apiVersion: v1
kind: Pod
metadata:
name: cgrouptest
spec:
initContainers:
- image: registry.cn-hangzhou.aliyuncs.com/denverdino/system-info
name: app
imagePullPolicy: IfNotPresent
command:
- "cp"
- "-r"
- "/system-info"
- "/app"
volumeMounts:
- mountPath: /app
name: app-volume
containers:
- image: tomcat:9-jre8
name: tomcat
imagePullPolicy: IfNotPresent
env:
- name: JAVA_OPTS
value: "-XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap"
volumeMounts:
- mountPath: /usr/local/tomcat/webapps
name: app-volume
ports:
- containerPort: 8080
resources:
requests:
memory: "256Mi"
cpu: "500m"
limits:
memory: "256Mi"
cpu: "500m"
volumes:
- name: app-volume
emptyDir: {}
$ kubectl create -f cgroup_test.yaml
pod "cgrouptest" created
$ kubectl exec cgrouptest curl http://localhost:8080/system-info/ | html2txt
Java version Oracle Corporation 1.8.0_162
Operating system Linux 4.9.64
Server Apache Tomcat/9.0.6
Memory Used 23 of 44 MB, Max 112 MB
Physica Memory 3951 MB
CPU Cores 2
**** Memory MXBean ****
Heap Memory Usage init = 8388608(8192K) used = 25280928(24688K) committed =
46661632(45568K) max = 117440512(114688K)
Non-Heap Memory Usage init = 2555904(2496K) used = 31970840(31221K) committed =
32768000(32000K) max = -1(-1K)

Perception of cgroup Resource Restrictions in Containers

If we cannot use the new features of JDK 8 and 9 (for example, if we are still using old JDK 6 applications), we can use a script in the container to obtain the container’s cgroup resource restrictions and use this to set the JVM heap size.

Conclusion

This article looks at a common heap setting problem arising when running Java applications in containers. Containers differ from virtual machines in that their resource restrictions are implemented using cgroups. Moreover, if internal container processes are not aware of the cgroup restrictions, memory and CPU allocation can produce resource conflicts and problems.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alibaba Cloud

Alibaba Cloud

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website:https://www.alibabacloud.com