Kubernetes Resource Quotas

1) Basic Quota : 2 Pods

nano myQuota.yamlapiVersion: v1
kind: ResourceQuota
metadata:
name: count-quotas
spec:
hard:
pods: "2"
kubectl create -f myQuota.yamlresourcequota/count-quotas created
kubectl get quotaNAME            CREATED AT
object-counts 2019-01-22T05:53:02Z
kubectl describe quota object-countsName:       object-counts
Namespace: default
Resource Used Hard
-------- ---- ----
pods 0 2
nano myQuota-Pod.yamlapiVersion: v1
kind: Pod
metadata:
name: quota-pod-1
spec:
containers:
- name: quota-container
image: busybox
imagePullPolicy: IfNotPresent

command: ['sh', '-c', 'echo Pod is Running ; sleep 3600']

restartPolicy: Never
terminationGracePeriodSeconds: 0
kubectl create -f myQuota-Pod.yamlpod/quota-pod-1 created
name: quota-pod-2
kubectl create -f myQuota-Pod.yamlpod/quota-pod-2 created
kubectl describe quota object-countsName:       object-counts
Namespace: default
Resource Used Hard
-------- ---- ----
pods 2 2
kubectl get poNAME          READY   STATUS    RESTARTS   AGE
quota-pod-1 1/1 Running 0 2m36s
quota-pod-2 1/1 Running 0 90s
name: quota-pod-3
kubectl create -f myQuota-Pod.yamlError from server (Forbidden): error when creating "myQuota-Pod.yaml": pods "quota-pod-3" is forbidden: exceeded quota: object-counts, requested: pods=1, used: pods=2, limited: pods=2
kubectl delete pod/quota-pod-1
pod "quota-pod-1" deleted
kubectl delete pod/quota-pod-2
pod "quota-pod-2" deleted
kubectl delete quota/object-counts
resourcequota "object-counts" deleted
kubectl get po
No resources found.

2) CPU Quota on Requests and Limits

nano myQuota.yamlapiVersion: v1
kind: ResourceQuota
metadata:
name: object-counts
spec:
hard:
requests.cpu: "1000m"
limits.cpu: "2000m"
kubectl create -f myQuota.yamlresourcequota/count-quotas created
nano myQuota-Pod-1.yamlapiVersion: v1
kind: Pod
metadata:
name: quota-pod-1
spec:
containers:
- name: quota-container
image: busybox
imagePullPolicy: IfNotPresent

command: ['sh', '-c', 'echo Pod is Running ; sleep 3600']
resources:
requests:
cpu: "500m"
limits:
cpu: "1500m"

restartPolicy: Never
terminationGracePeriodSeconds: 0
nano myQuota-Pod-2.yamlapiVersion: v1
kind: Pod
metadata:
name: quota-pod-2
spec:
containers:
- name: quota-container
image: busybox
imagePullPolicy: IfNotPresent

command: ['sh', '-c', 'echo Pod is Running ; sleep 3600']
resources:
requests:
cpu: "500m"
limits:
cpu: "500m"

restartPolicy: Never
terminationGracePeriodSeconds: 0
nano myQuota-Pod-3.yamlapiVersion: v1
kind: Pod
metadata:
name: quota-pod-3
spec:
containers:
- name: quota-container
image: busybox
imagePullPolicy: IfNotPresent

command: ['sh', '-c', 'echo Pod is Running ; sleep 3600']
resources:
requests:
cpu: "10m"
limits:
cpu: "50m"

restartPolicy: Never
terminationGracePeriodSeconds: 0
kubectl create -f myQuota-Pod-1.yaml
pod/quota-pod-1 created
kubectl create -f myQuota-Pod-2.yaml
pod/quota-pod-2 created
kubectl describe quota object-countsName:         object-counts
Namespace: default
Resource Used Hard
-------- ---- ----
limits.cpu 2 2
requests.cpu 1 1
kubectl create -f myQuota-Pod-3.yamlError from server (Forbidden): error when creating "myQuota-Pod-3.yaml": pods "quota-pod-3" is forbidden: exceeded quota: object-counts, requested: limits.cpu=50m,requests.cpu=10m, used: limits.cpu=2,requests.cpu=1, limited: limits.cpu=2,requests.cpu=1
kubectl delete -f myQuota-Pod-1.yaml
pod "quota-pod-1" deleted
kubectl delete -f myQuota-Pod-2.yaml
pod "quota-pod-2" deleted
kubectl delete quota/object-counts
resourcequota "object-counts" deleted

3) Quota Count/ Syntax

nano myQuota.yamlapiVersion: v1
kind: ResourceQuota
metadata:
name: object-counts
spec:
hard:
secrets: "2"
kubectl create -f myQuota.yamlresourcequota/count-quotas created
kubectl describe quota object-counts
Name: object-counts
Namespace: default
Resource Used Hard
-------- ---- ----
secrets 1 2
kubectl get secretsNAME                  TYPE                                  DATA   AGE
default-token-gs2wt kubernetes.io/service-account-token 3 26d
kubectl create secret generic my-insecure-secret-1 --from-literal=literalkey1=insecure-secret-value-1
secret/my-insecure-secret-1 created
kubectl get secretsNAME                   TYPE                                  DATA   AGE
default-token-gs2wt kubernetes.io/service-account-token 3 26d
my-insecure-secret-1 Opaque 1 3s
kubectl describe quota object-countsName:       object-counts
Namespace: default
Resource Used Hard
-------- ---- ----
secrets 2 2
kubectl create secret generic my-insecure-secret-2 --from-literal=literalkey2=insecure-secret-value-2
Error from server (Forbidden): secrets "my-insecure-secret-2" is forbidden: exceeded quota: object-counts, requested: secrets=1, used: secrets=2, limited: secrets=2
kubectl delete secret/my-insecure-secret-1
secret "my-insecure-secret-1" deleted
kubectl delete quota/object-counts
nano myQuota.yamlapiVersion: v1
kind: ResourceQuota
metadata:
name: object-counts
spec:
hard:
count/secrets: "2"
kubectl create -f myQuota.yamlresourcequota/count-quotas created
kubectl describe quota object-countsName:          object-counts
Namespace: default
Resource Used Hard
-------- ---- ----
count/secrets 1 2
kubectl get secretsNAME                  TYPE                                  DATA   AGE
default-token-gs2wt kubernetes.io/service-account-token 3 26d
spec:
hard:
pods: "2"
secrets: "2"
spec:
hard:
count/pods: "2"
count/secrets: "2"
kubectl delete -f myQuota.yamlresourcequota "counts-quota" deleted

4) Quota on Quality of Service (QoS) Class

For a Pod to be given a QoS class of BestEffort , the Containers in the Pod must not have any memory or CPU limits or requests.

nano myQuota.yamlapiVersion: v1
kind: ResourceQuota
metadata:
name: object-counts
spec:
hard:
count/pods: "2"
scopes:
- BestEffort
kubectl create -f myQuota.yamlresourcequota/count-quotas created
kubectl describe quota object-countsName:       object-counts
Namespace: default
Scopes: BestEffort
* Matches all pods that do not have resource requirements set. These pods have a best effort quality of service.
Resource Used Hard
-------- ---- ----
count/pods 0 2
resources:
requests:
cpu: "500m"
limits:
cpu: "500m"
nano myQuota-Pod-1.yamlapiVersion: v1
kind: Pod
metadata:
name: quota-pod-1
spec:
containers:
- name: quota-container
image: busybox
imagePullPolicy: IfNotPresent

command: ['sh', '-c', 'echo Pod is Running ; sleep 3600']

restartPolicy: Never
terminationGracePeriodSeconds: 0
nano myQuota-Pod-2.yamlapiVersion: v1
kind: Pod
metadata:
name: quota-pod-2
spec:
containers:
- name: quota-container
image: busybox
imagePullPolicy: IfNotPresent

command: ['sh', '-c', 'echo Pod is Running ; sleep 3600']

restartPolicy: Never
terminationGracePeriodSeconds: 0
nano myQuota-Pod-3.yamlapiVersion: v1
kind: Pod
metadata:
name: quota-pod-3
spec:
containers:
- name: quota-container
image: busybox
imagePullPolicy: IfNotPresent

command: ['sh', '-c', 'echo Pod is Running ; sleep 3600']

restartPolicy: Never
terminationGracePeriodSeconds: 0
kubectl create -f myQuota-Pod-1.yaml
pod/quota-pod-1 created
kubectl describe quota object-counts
Name: object-counts
Namespace: default
Scopes: BestEffort
* Matches all pods that do not have resource requirements set. These pods have a best effort quality of service.
Resource Used Hard
-------- ---- ----
count/pods 1 2
kubectl create -f myQuota-Pod-2.yaml
pod/quota-pod-2 created
kubectl describe quota object-countsName:       object-counts
Namespace: default
Scopes: BestEffort
* Matches all pods that do not have resource requirements set. These pods have a best effort quality of service.
Resource Used Hard
-------- ---- ----
count/pods 2 2
kubectl create -f myQuota-Pod-3.yaml
Error from server (Forbidden): error when creating "myQuota-Pod-3.yaml": pods "quota-pod-3" is forbidden: exceeded quota: object-counts, requested: count/pods=1, used: count/pods=2, limited: count/pods=2
kubectl describe pod/quota-pod-1|grep Best
QoS Class: BestEffort
kubectl describe pod/quota-pod-2|grep Best
QoS Class: BestEffort
kubectl delete -f myQuota-Pod-1.yaml
pod "quota-pod-1" deleted
kubectl delete -f myQuota-Pod-2.yaml
pod "quota-pod-2" deleted
kubectl delete quota/object-counts
resourcequota "object-counts" deleted

5) Scopes: NotTerminating Quotas

Matches all pods that do not have an active deadline. These pods usually include long running pods whose container command is not expected to terminate.

nano myQuota.yamlapiVersion: v1
kind: ResourceQuota
metadata:
name: object-counts
spec:
hard:
count/pods: "2"
scopes:
- NotTerminating
kubectl create -f myQuota.yaml
resourcequota/object-counts created
kubectl describe quota object-countsName: object-counts
Namespace: default
Scopes: NotTerminating
* Matches all pods that do not have an active deadline. These pods usually include long running pods whose container command is not expected to terminate.
Resource Used Hard
-------- ---- ----
count/pods 0 2
kubectl create -f myQuota-Pod-1.yaml
pod/quota-pod-1 created
kubectl describe quota object-countsName: object-counts
Namespace: default
Scopes: NotTerminating
* Matches all pods that do not have an active deadline. These pods usually include long running pods whose container command is not expected to terminate.
Resource Used Hard
-------- ---- ----
count/pods 1 2
kubectl get po
NAME READY STATUS RESTARTS AGE
quota-pod-1 1/1 Running 0 10s
kubectl create -f myQuota-Pod-2.yaml
pod/quota-pod-2 created
kubectl get poNAME READY STATUS RESTARTS AGE
quota-pod-1 1/1 Running 0 20s
quota-pod-2 1/1 Running 0 4s
kubectl describe quota object-counts
Name: object-counts
Namespace: default
Scopes: NotTerminating
* Matches all pods that do not have an active deadline. These pods usually include long running pods whose container command is not expected to terminate.
Resource Used Hard
-------- ---- ----
count/pods 2 2
kubectl create -f myQuota-Pod-3.yaml
Error from server (Forbidden): error when creating "myQuota-Pod-3.yaml": pods "quota-pod-3" is forbidden: exceeded quota: object-counts, requested: count/pods=1, used: count/pods=2, limited: count/pods=2
kubectl delete quota/object-counts
resourcequota "object-counts" deleted
kubectl delete -f myQuota-Pod-1.yaml
pod "quota-pod-1" deleted
kubectl delete -f myQuota-Pod-2.yaml
pod "quota-pod-2" deleted

6) Quotas Need Value in YAML Spec

If the quota has a value specified for requests.cpu or requests.memory, then it requires that every incoming container makes an explicit request for those resources.

If the quota has a value specified for limits.cpu or limits.memory, then it requires that every incoming container specifies an explicit limit for those resources

nano myQuota.yamlapiVersion: v1
kind: ResourceQuota
metadata:
name: object-counts
spec:
hard:
requests.cpu: "1000m"
limits.cpu: "2000m"
count/pods: "2"
kubectl create -f myQuota.yamlresourcequota/count-quotas created
kubectl describe quota object-countsName:         object-counts
Namespace: default
Resource Used Hard
-------- ---- ----
count/pods 0 2
limits.cpu 0 2
requests.cpu 0 1
nano myQuota-Pod-1.yamlapiVersion: v1
kind: Pod
metadata:
name: quota-pod-1
spec:
containers:
- name: quota-container
image: busybox
imagePullPolicy: IfNotPresent

command: ['sh', '-c', 'echo Pod is Running ; sleep 3600']

restartPolicy: Never
terminationGracePeriodSeconds: 0
kubectl create -f myQuota-Pod-1.yamlError from server (Forbidden): error when creating "myQuota-Pod-1.yaml": pods "quota-pod-1" is forbidden: failed quota: object-counts: must specify limits.cpu,requests.cpu
kubectl delete -f myQuota.yamlresourcequota "counts-quota" deleted

7) Resource Quota Per PriorityClass

8) Storage Resource Quota

You can limit the total sum of storage resources that can be requested in a given namespace.

In addition, you can limit consumption of storage resources based on associated storage-class.

9) Quota and Cluster Capacity

ResourceQuotas are independent of the cluster capacity. They are expressed in absolute units.

So, if you add nodes to your cluster, this does not automatically give each namespace the ability to consume more resources.

Note that resource quota divides up aggregate cluster resources, but it creates no restrictions around nodes : pods from several namespaces may run on the samenode .

10) Start Using Quotas

Original Source

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store