Kubernetes CronJobs — Part 1: Basics

By Alwyn Botha, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud’s incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.

Kubernetes cron jobs are used to define cron jobs.

From https://en.wikipedia.org/wiki/Cron

Use cron to schedule jobs (commands or shell scripts) to run periodically at fixed times, dates, or intervals.

It typically automates system maintenance or administration — though its general-purpose nature makes it useful for things like downloading files from the Internet and downloading email at regular intervals.

Cron is most suitable for scheduling repetitive tasks.

Kubernetes cron jobs have the features of Linux cron jobs, plus a few unique features of its own.

Part 1 introduces you to Kubernetes cron job basics plus some of these unique features.

1) Basic Cron Job Example

15 lines below you will find our most basic cron job YAML spec:

kind: CronJob for cron jobs.

Note the schedule: /1 *

It specifies when this cron job must run. Time syntax identical to what are used to on Linux systems.

Our basic cron job will run every minute.

schedule fields: minute hour day-of-month month day-of-week

*/1 … means every minute

Rest of fields: every hour, every day, every day-of-month, every month, every day-of-week

If you run man crontab in your terminal you will get comprehensive detail about the crontab time formats allowed.

( All cron jobs in this tutorial run every minute so that we can experiment with all cron job functionality as quickly as possible. It also runs every day, …. and so on so that anyone that runs it will get a running cron job ANY day of the year )

concurrencyPolicy: Allow is the default — we will learn and experiment with this later.

The rest of the YAML spec content is identical for a Pod.

nano myCronJob.yamlapiVersion: batch/v1beta1
kind: CronJob
metadata:
name: mycronjob
spec:
schedule: "*/1 * * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: mycron-container
image: alpine
imagePullPolicy: IfNotPresent

command: ['sh', '-c', 'echo Job Pod is Running ; sleep 5']

restartPolicy: OnFailure
terminationGracePeriodSeconds: 0

concurrencyPolicy: Allow

Create the cron job:

kubectl create -f myCronJob.yaml
cronjob.batch/mycronjob created

Unfortunately Linux cron job functionality allows per minute as smallest time increments.

So throughout this tutorial we have to repeatedly wait around a minute before something happens.

You can use the date command repeately to peek as to when the seconds digits will be 00 — on those 00s cron job starts our cron jobs in this tutorial.

Alternatively use

  • kubectl get job
  • kubectl get pods

repeatedly until you see some action.

kubectl get cronjobNAME        SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
mycronjob */1 * * * * False 0 <none> 14s
kubectl get jobNo resources found.

kubectl get cronjob provides a summary overview. Most of the time it is quite useless ( since there is no detail … and summary detail that would have been cool to have, is not there ).

After around a minute:

kubectl get job
NAME COMPLETIONS DURATION AGE
mycronjob-1548664260 0/1 4s 4s
kubectl get po
NAME READY STATUS RESTARTS AGE
mycronjob-1548664260-7dhlm 1/1 Running 0 5s

These 2 commands show different details. In this tutorial we will always use both these commands to learn when which helps most.

kubectl get job shows the JOB has been running 4s. COMPLETIONS is zero … it is still running.

kubectl get po shows we have one READY ( Running ) Pod … been running 5s.

Note jobs get prefixed with our cron job name ( from YAML spec )

Note Pods for that job get prefixed with our cron job name ( in YAML spec ) + 5 character hash ( random characters ).

Mere seconds later.

kubectl get jobNAME                   COMPLETIONS   DURATION   AGE
mycronjob-1548664260 1/1 6s 6s
kubectl get poNAME READY STATUS RESTARTS AGE
mycronjob-1548664260-7dhlm 0/1 Completed 0 7s

Job only had to sleep 5 seconds.

Using the combined information from both commands:

  • job has completed: COMPLETIONS 1
  • job took 6s … DURATION
  • Pod is Completed
  • Pod is no longer available … READY is 0

This is the look of a successfully ran cron job.

More information :

kubectl get cronjobNAME        SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
mycronjob */1 * * * * False 0 54s 87s

We will use SUSPEND later.

ACTIVE 0 … no Pods currently busy processing work for this cron job.

We saw moments ago our Pod slept 5 seconds and is now completed. For the rest of the minute the ACTIVE column will stay zero. It is awaiting the next precise minute to start another run of this cron job.

You will see around 30 cron job runs in this tutorial … understanding will emerge.

Nearly a minute later … all that happened is AGE increased … awaiting the next precise minute to start another run of this cron job.

kubectl get job
NAME COMPLETIONS DURATION AGE
mycronjob-1548664260 1/1 6s 56s
kubectl get po
mycronjob-1548664260-7dhlm 0/1 Completed 0 56s

30 seconds later.

kubectl get cronjob
NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE
mycronjob */1 * * * * False 1 1s 94s

Active is ONE. A Pod for this cron job is running.

kubectl get jobNAME                   COMPLETIONS   DURATION   AGE
mycronjob-1548664260 1/1 6s 63s
mycronjob-1548664320 0/1 3s 3s

We see our completed job listed first and new running job listed second — hint — look at AGE.

kubectl get po
NAME READY STATUS RESTARTS AGE
mycronjob-1548664260-7dhlm 0/1 Completed 0 63s
mycronjob-1548664320-75n9z 1/1 Running 0 3s

We see our completed POD first and new running POD last — hint — look at AGE.

10 seconds later: the second job and Pod are completed. ( It only had to sleep 5 seconds )

kubectl get jobNAME                   COMPLETIONS   DURATION   AGE
mycronjob-1548664260 1/1 6s 71s
mycronjob-1548664320 1/1 6s 11s
kubectl get poNAME READY STATUS RESTARTS AGE
mycronjob-1548664260-7dhlm 0/1 Completed 0 71s
mycronjob-1548664320-75n9z 0/1 Completed 0 11s

No new job is running — awaits next scheduled minute.

Another way to gather information about our cron job : ( irrelevant fields hidden )

kubectl describe cronjob/mycronjobName:                       mycronjob
Schedule: */1 * * * *
Concurrency Policy: Allow
Pod Template:
Containers:
mycron-container:
Command:
echo Job Pod is Running ; sleep 5
Last Schedule Time: Mon, 28 Jan 2019 10:33:00 +0200
Active Jobs: mycronjob-1548664380
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulCreate 2m6s cronjob-controller Created job mycronjob-1548664260
Normal SawCompletedJob 116s cronjob-controller Saw completed job: mycronjob-1548664260
Normal SuccessfulCreate 66s cronjob-controller Created job mycronjob-1548664320
Normal SawCompletedJob 56s cronjob-controller Saw completed job: mycronjob-1548664320
Normal SuccessfulCreate 6s cronjob-controller Created job mycronjob-1548664380

In events we see a new job created every minute.

We also see the … Saw completed job: … around 10 seconds after it started.

After 3 minutes we have 3 completed jobs and Pods.

kubectl get job
NAME COMPLETIONS DURATION AGE
mycronjob-1548664560 1/1 7s 2m53s
mycronjob-1548664620 1/1 6s 112s
mycronjob-1548664680 1/1 7s 52s
kubectl get pods
NAME READY STATUS RESTARTS AGE
mycronjob-1548664560-nq4zm 0/1 Completed 0 2m53s
mycronjob-1548664620-5srx2 0/1 Completed 0 112s
mycronjob-1548664680-mxpdx 0/1 Completed 0 52s

Seconds later, we JUST caught the creation of Pod number four.

kubectl get job
NAME COMPLETIONS DURATION AGE
mycronjob-1548664560 1/1 7s 3m3s
mycronjob-1548664620 1/1 6s 2m2s
mycronjob-1548664680 1/1 7s 62s
mycronjob-1548664740 0/1 2s 2s
kubectl get pods
NAME READY STATUS RESTARTS AGE
mycronjob-1548664560-nq4zm 0/1 Completed 0 3m3s
mycronjob-1548664620-5srx2 0/1 Completed 0 2m2s
mycronjob-1548664680-mxpdx 0/1 Completed 0 62s
mycronjob-1548664740-cqrlt 0/1 ContainerCreating 0 2s

This specific output makes it VERY easy to see in AGE … one new Pod every minute.

kubectl get cronjobNAME        SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
mycronjob */1 * * * * False 1 12s 7m45s

It would have been great if this command also showed:

  • number of jobs successfully completed
  • number of jobs UNsuccessfully completed

Introduction example complete. Delete job:

kubectl delete -f myCronJob.yaml 

cronjob.batch "mycronjob" deleted

This deletes the job, all the Pods that ran under its management and all their job logs.

2) concurrencyPolicy: Allow

From https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#concurrency-policy

It specifies how to treat concurrent executions of a job that is created by this cron job.

Allow (default): The cron job allows concurrently running jobs

Our previous example was kept simple. Sleep for 5 seconds.

The concurrencyPolicy was irrelevant in its case: no two jobs / Pods were ever able to run simultaneously.

This example will demonstrate concurrencyPolicy in action.

Every minute a new job must run.

Our Pod must sleep for 90 seconds.

So after 60 seconds — when Pod still has 30 seconds of sleeping work left — another Pod will be automatically scheduled.

concurrencyPolicy: Allow will allow this existing Pod to continue running for the full 90 seconds.

The policy will also allow the simultaneous starting and running of the second newly scheduled Pod.

nano myCronJob.yamlapiVersion: batch/v1beta1
kind: CronJob
metadata:
name: mycronjob
spec:
schedule: "*/1 * * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: mycron-container
image: alpine
imagePullPolicy: IfNotPresent

command: ['sh', '-c', 'echo Job Pod is Running ; sleep 90']

restartPolicy: OnFailure
terminationGracePeriodSeconds: 0

concurrencyPolicy: Allow
kubectl create -f myCronJob.yaml

cronjob.batch/mycronjob created

Run:

kubectl get cronjob
NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE
mycronjob */1 * * * * False 0 <none> 15s

Nothing informative.

Many seconds later, first Pod starts:

kubectl get job
mycronjob-1548665220 0/1 4s 4s
kubectl get pods
NAME READY STATUS RESTARTS AGE
mycronjob-1548665220-8pdnw 1/1 Running 0 4s

Important … see below: One minute later, new Pod starts up, and old one continues to run.

Note the old job is still running, its COMPLETIONS is still zero.

The old pod is also READY … it is still running.

kubectl get job
NAME COMPLETIONS DURATION AGE
mycronjob-1548665220 0/1 63s 63s
mycronjob-1548665280 0/1 3s 3s
kubectl get pods
NAME READY STATUS RESTARTS AGE
mycronjob-1548665220-8pdnw 1/1 Running 0 64s
mycronjob-1548665280-6cnjm 1/1 Running 0 4s

Two jobs, each with one Pod is running simultaneously.

kubectl get job
NAME COMPLETIONS DURATION AGE
mycronjob-1548665220 0/1 76s 76s
mycronjob-1548665280 0/1 16s 16s
kubectl get pods
NAME READY STATUS RESTARTS AGE
mycronjob-1548665220-8pdnw 1/1 Running 0 76s
mycronjob-1548665280-6cnjm 1/1 Running 0 16s

After 90 seconds total runtime.

kubectl get job
NAME COMPLETIONS DURATION AGE
mycronjob-1548665220 1/1 92s 94s
mycronjob-1548665280 0/1 34s 34s
kubectl get pods
NAME READY STATUS RESTARTS AGE
mycronjob-1548665220-8pdnw 0/1 Completed 0 94s
mycronjob-1548665280-6cnjm 1/1 Running 0 34s

Above: first job is complete, second job is running.

Below: after several minutes: one job started every minute.

First 3 jobs show COMPLETIONS equal to 1 ( success ). DURATION correct for completed jobs.

kubectl get job
NAME COMPLETIONS DURATION AGE
mycronjob-1548665280 1/1 92s 4m2s
mycronjob-1548665340 1/1 91s 3m1s
mycronjob-1548665400 1/1 92s 2m1s
mycronjob-1548665460 0/1 61s 61s
mycronjob-1548665520 0/1 1s 1s

History of completed Pods.

kubectl get pods
NAME READY STATUS RESTARTS AGE
mycronjob-1548665280-6cnjm 0/1 Completed 0 4m2s
mycronjob-1548665340-wm2zk 0/1 Completed 0 3m1s
mycronjob-1548665400-nnfgk 0/1 Completed 0 2m1s
mycronjob-1548665460-crn6x 1/1 Running 0 61s
mycronjob-1548665520-qt7zd 0/1 ContainerCreating 0 1s

3 completed pods are no READY anymore. ( Completed successfully )

One pod has been running 61 seconds and is still ready. It still has 29 seconds of sleeping to do.

Our cron job precise minute timer kicked off another Pod that is still in preliminary ContainerCreating state.

Reference : https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#jobs-history-limits

By default 3 successfully done jobs are kept.

Minutes later … we can see 2 old jobs deleted ( near bottom )

kubectl describe cronjob/mycronjobEvents:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulCreate 5m44s cronjob-controller Created job mycronjob-1548665220
Normal SuccessfulCreate 4m44s cronjob-controller Created job mycronjob-1548665280
Normal SawCompletedJob 4m3s cronjob-controller Saw completed job: mycronjob-1548665220
Normal SuccessfulCreate 3m43s cronjob-controller Created job mycronjob-1548665340
Normal SawCompletedJob 3m3s cronjob-controller Saw completed job: mycronjob-1548665280
Normal SuccessfulCreate 2m43s cronjob-controller Created job mycronjob-1548665400
Normal SawCompletedJob 2m3s cronjob-controller Saw completed job: mycronjob-1548665340
Normal SuccessfulCreate 103s cronjob-controller Created job mycronjob-1548665460
Normal SawCompletedJob 63s cronjob-controller Saw completed job: mycronjob-1548665400

Normal SuccessfulDelete 63s cronjob-controller Deleted job mycronjob-1548665220 ..........

Normal SuccessfulCreate 43s cronjob-controller Created job mycronjob-1548665520
Normal SawCompletedJob 3s cronjob-controller Saw completed job: mycronjob-1548665460

Normal SuccessfulDelete 3s cronjob-controller Deleted job mycronjob-1548665280 ...........

Oldest jobs not shown. Only last 3 plus currently running ones.

kubectl get pods
NAME READY STATUS RESTARTS AGE
mycronjob-1548665400-nnfgk 0/1 Completed 0 4m21s
mycronjob-1548665460-crn6x 0/1 Completed 0 3m21s
mycronjob-1548665520-qt7zd 0/1 Completed 0 2m21s
mycronjob-1548665580-qzpqq 1/1 Running 0 81s
mycronjob-1548665640-68dhw 1/1 Running 0 21s

Demo complete, delete …

kubectl delete -f myCronJob.yaml 

cronjob.batch "mycronjob" deleted

IMPORTANT : concurrencyPolicy: Allow … Your cron jobs MUST be able to have functionality that enables more than one job to be running simultaneously. Probable file locking, startup and cleanup considerations.

3) concurrencyPolicy: Forbid

From https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#concurrency-policy

Forbid: The cron job does not allow concurrent runs; if it is time for a new job run and the previous job run hasn’t finished yet, the cron job skips the new job run.

Notice last line in YAML spec below.

This demo will see Forbid in action.

nano myCronJob.yamlapiVersion: batch/v1beta1
kind: CronJob
metadata:
name: mycronjob
spec:
schedule: "*/1 * * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: mycron-container
image: alpine
imagePullPolicy: IfNotPresent

command: ['sh', '-c', 'echo Job Pod is Running ; sleep 90']

restartPolicy: OnFailure
terminationGracePeriodSeconds: 0

concurrencyPolicy: Forbid

Create the Job

kubectl create -f myCronJob.yaml 

cronjob.batch/mycronjob created

After a minute:

kubectl get job
NAME COMPLETIONS DURATION AGE
mycronjob-1548666360 0/1 57s 57s
kubectl get pods
NAME READY STATUS RESTARTS AGE
mycronjob-1548666360-gg26f 1/1 Running 0 57s

Below: one minute elapsed, original first job still running.

New job forbidden to start. ( We do not see a new job with age of around 18 seconds )

kubectl get job
NAME COMPLETIONS DURATION AGE
mycronjob-1548666360 0/1 78s 78s
NAME READY STATUS RESTARTS AGE
mycronjob-1548666360-gg26f 1/1 Running 0 78s

Nearly 90 seconds later — first job busy with last seconds of running.

kubectl get job
NAME COMPLETIONS DURATION AGE
mycronjob-1548666360 1/1 92s 95s
kubectl get pods
NAME READY STATUS RESTARTS AGE
mycronjob-1548666360-gg26f 0/1 Completed 0 95s

First job complete, NOW second job starts immediately.

NAME                   COMPLETIONS   DURATION   AGE
mycronjob-1548666360 1/1 92s 103s
mycronjob-1548666420 0/1 3s 3s
kubectl get pods
NAME READY STATUS RESTARTS AGE
mycronjob-1548666360-gg26f 0/1 Completed 0 103s
mycronjob-1548666420-t92nc 1/1 Running 0 3s

90 seconds later … second job completed.

kubectl get job
NAME COMPLETIONS DURATION AGE
mycronjob-1548666360 1/1 92s 3m16s
mycronjob-1548666420 1/1 93s 96s
kubectl get pods
NAME READY STATUS RESTARTS AGE
mycronjob-1548666360-gg26f 0/1 Completed 0 3m16s
mycronjob-1548666420-t92nc 0/1 Completed 0 96s

Mere seconds later : third job starts.

kubectl get job
NAME COMPLETIONS DURATION AGE
mycronjob-1548666360 1/1 92s 3m21s
mycronjob-1548666420 1/1 93s 101s
mycronjob-1548666540 0/1 0s
kubectl get pods
NAME READY STATUS RESTARTS AGE
mycronjob-1548666360-gg26f 0/1 Completed 0 3m21s
mycronjob-1548666420-t92nc 0/1 Completed 0 101s
mycronjob-1548666540-zds5w 0/1 ContainerCreating 0 0s

8 seconds later … third job still busy.

kubectl get job
NAME COMPLETIONS DURATION AGE
mycronjob-1548666360 1/1 92s 3m29s
mycronjob-1548666420 1/1 93s 109s
mycronjob-1548666540 0/1 8s 8s
kubectl get pods
NAME READY STATUS RESTARTS AGE
mycronjob-1548666360-gg26f 0/1 Completed 0 3m29s
mycronjob-1548666420-t92nc 0/1 Completed 0 109s
mycronjob-1548666540-zds5w 1/1 Running 0 8s

kubectl describe output :

kubectl describe cronjob/mycronjobEvents:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulCreate 3m34s cronjob-controller Created job mycronjob-1548666360

Normal SawCompletedJob 114s cronjob-controller Saw completed job: mycronjob-1548666360
Normal SuccessfulCreate 114s cronjob-controller Created job mycronjob-1548666420

Normal SawCompletedJob 13s cronjob-controller Saw completed job: mycronjob-1548666420
Normal SuccessfulCreate 13s cronjob-controller Created job mycronjob-1548666540

Note the original neat schedule of one new job per minute is now destroyed.

Every new job only start when previous one completes. Not on the minute, but 30 seconds later since each job runs 90 seconds.

Use concurrencyPolicy: Forbid when you want each cron job to fully complete ALL BY ITSELF, then start the next job in the schedule queue. Subsequent cron job start times depend on runtime of previous job, not on precise schedule.

kubectl get cronjob does not show any detail of all this that just happened.

kubectl get cronjob
NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE
mycronjob */1 * * * * False 1 65s 12m

Demo done, delete …

kubectl delete -f myCronJob.yaml 

cronjob.batch "mycronjob" deleted

4) concurrencyPolicy: Replace

From https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#concurrency-policy

Replace : If it is time for a new job run and the previous job run hasn’t finished yet, the cron job replaces the currently running job run with a new job run

Notice last line in YAML spec below.

This demo will see Replace in action.

nano myCronJob.yaml

apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: mycronjob
spec:
schedule: "*/1 * * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: mycron-container
image: alpine
imagePullPolicy: IfNotPresent

command: ['sh', '-c', 'echo Job Pod is Running ; sleep 90']

restartPolicy: OnFailure
terminationGracePeriodSeconds: 0

concurrencyPolicy: Replace
kubectl create -f myCronJob.yaml

cronjob.batch/mycronjob created

Previous job after 60 seconds.

kubectl get job
NAME COMPLETIONS DURATION AGE
mycronjob-1548667320 0/1 58s 58s
kubectl get pods
NAME READY STATUS RESTARTS AGE
mycronjob-1548667320-68bjn 1/1 Running 0 58s

New job started. Replace works. It replaced the running job that wanted to sleep for 90 seconds.

kubectl get job
NAME COMPLETIONS DURATION AGE
mycronjob-1548667380 0/1 6s 6s
kubectl get pods
NAME READY STATUS RESTARTS AGE
mycronjob-1548667380-l7vvx 1/1 Running 0 6s

We can see in events below that after a job runs for 60 seconds it gets deleted. New one takes its place.

kubectl describe cronjob/mycronjob
Concurrency Policy: Replace
Pod Template:
Containers:
mycron-container:
Command:
echo Job Pod is Running ; sleep 90
Events:
---- ------ ---- ---- -------
Normal SuccessfulCreate 77s cronjob-controller Created job mycronjob-1548667320
Normal SuccessfulDelete 17s cronjob-controller Deleted job mycronjob-1548667320

Second job gets same treatment: replacement after 60 seconds.

kubectl get job
NAME COMPLETIONS DURATION AGE
mycronjob-1548667380 0/1 52s 52s
kubectl get pods
NAME READY STATUS RESTARTS AGE
mycronjob-1548667380-l7vvx 1/1 Running 0 53s
... 8 seconds later ... NEW job ...kubectl get job
NAME COMPLETIONS DURATION AGE
mycronjob-1548667440 0/1 1s 1s
kubectl get pods
NAME READY STATUS RESTARTS AGE
mycronjob-1548667440-t95z5 1/1 Running 0 1s

Summary in events:

Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulCreate 2m17s cronjob-controller Created job mycronjob-1548667320
Normal SuccessfulDelete 77s cronjob-controller Deleted job mycronjob-1548667320

Normal SuccessfulCreate 77s cronjob-controller Created job mycronjob-1548667380
Normal SuccessfulDelete 16s cronjob-controller Deleted job mycronjob-1548667380

Normal SuccessfulCreate 16s cronjob-controller Created job mycronjob-1548667440

Every job only runs for 60 seconds maximum. It gets replaced by new job starting every minute.

Use the logic of concurrencyPolicy: Replace when it is appropriate for your production cron job.

kubectl delete -f myCronJob.yaml 

cronjob.batch "mycronjob" deleted

Original Source

https://www.alibabacloud.com/blog/kubernetes-cronjobs---part-1-basics_595021?spm=a2c41.13112140.0.0

Written by

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store