introduction
A Kubernetes cluster consists of
multiple node machines that host pods, the smallest units in the Kubernetes architecture. A cluster administrator defines the desired state with the required number of pod replicas. Consequently, Kubernetes ensures that the requested number of pods is always up and running.
However, when managing a Kubernetes cluster, certain tasks require pods to finish after completion. To perform those tasks, administrators use specific workload resources called jobs.
This tutorial will show you how to create, manage, and delete Kubernetes jobs.
Prerequisites
A Kubernetes
- cluster (for testing purposes, you can install Minikube).
- Kubectl installed.
What is Kubernetes
Job?
A
Kubernetes job is a workload controller object that performs one or more finite tasks on a cluster. The finite nature of jobs differentiates them from most controller objects, such as implementations, replica sets, stateful sets, and daemon sets.
Although these objects permanently maintain the desired state and number of pods in the cluster, the jobs
run until they complete the task and then terminate the associated pods.
Kubernetes job use cases
Kubernetes jobs can perform many important tasks in a cluster, including:
Maintenance tasks (such as
- performing backups).
- Big calculations
- Batch tasks (such as sending emails).
- Monitoring of node behaviors
- Work queue management
- Some Helm charts use jobs to install applications.
.
.
.
How to create
a Kubernetes job
A job in Kubernetes is created using a YAML file. Follow the steps below to deploy a Kubernetes job.
1. Create a YAML file using a text editor.
nano [file name].yaml
The file provides the settings needed for the job
.
2. The following example creates a test job file. This job runs a pod with a container based on the alpine:latest Docker image. Inside the container, the job prints the numbers from one to nine and then finishes the container.
apiVersion: batch/v1 kind: Job metadata: name: test-job spec: template: metadata: name: test-job spec: containers: – name: test image: alpine:latest command: – “bin/sh” – “-c” – “for i in 1 2 3 4 5 6 7 8 9 ; echo $i ; done” restartPolicy: Never
save and exit the file
. 3. Apply the YAML file with kubectl
: kubectl
apply -f [file name].yaml
The result confirms the successful creation of the job
.
4. Confirm the job execution by checking the status of the pods in the cluster
: kubectl get pod
The READY column shows that the pod is no longer running and the pod status is Completed
. Use kubectl describes
to view pod details: kubectl describes
pod [pod-name] The output shows the
pod status as OK
.
The containers in the output shows the status of the container as Finished. The reason for termination is the completion of the operation.
You can also check the job itself by typing the following command:
kubectl get job
The COMPLETIONS column shows that the job completed successfully
. Kubernetes job management You can configure a
Kubernetes job
to run once or multiple times. Multiple instances of a single job can run successively or simultaneously.
Run
job more than once
To configure a job to perform the same task more than once, add the Completions field in the specifications section of the YAML manifest.
apiVersion: batch/v1 kind: Job metadata: name: test-job spec: completions: 10 template: metadata: name: test-job spec: containers: – name: test image: alpine:latest command: – “bin/sh” – “-c” – “for i in 1 2 3 4 5 6 7 8 9 ; echo $i ; done” restartPolicy: Never After applying the above YAML, use the -watch mark with kubectl get
to monitor job completion in real time. kubectl get job -watch
When configured in this way, the job completes a single task instance and starts another until it reaches the number of completions provided in the YAML file
.
Run parallel job instances
Kubernetes can run more than one task instance at the same time. With sufficient resources, this action improves the speed of job completion.
To use this feature, enter the number of tasks you want the system to run simultaneously in the spec.parallelism field of the YAML file.
The following example defines a job that runs ten times, with five instances running simultaneously.
apiVersion: batch/v1 kind: Job metadata: name: test-job spec: Terminations: 10 Parallelism: 5 Template: Metadata: Name: Test job specification: Containers: – Name: Test image: Alpine:Last command: – “bin/sh” – “-c” – “for I in 1 2 3 4 5 6 7 8 9 ; echo $i ; done” restartPolicy: Never
After applying the YAML file, check the progress of
the job: kubectl get job -watch
The result shows that the task was performed much faster with the included parallelism option:
Limit job completion time
Include the spec.activeDeadlineSeconds field in the YAML manifesto of work to limit the duration of work. The numeric value corresponds to the number of seconds after which the job ends, regardless of whether it was fully performed.
The following example shows a job that will repeat ten times, unless it exceeds the ten-second limit
: apiVersion: batch/v1 type: Job metadata: name: test job specification: terminations: 10 activeDeadlineSeconds: 10 template: metadata: name: test job specification: containers: – name: test image: alpine:last command: – “bin/sh” – “-c” – “for i in 1 2 3 4 5 6 7 8 9 ; echo $i ; done” restartPolicy: Never
save and apply the file. After some time, check
the status of the pods: kubectl get pod The output shows
only three completed pods
.
Check job status
: kubectl get job -watch The
result shows the last instance of work completed at the ten-second mark
.
To see why the job stopped running, view the job details by typing:
kubectl describes the job [job name]
The reason provided in the command output is DeadlineExceeded
.
Schedule work
Use CronJobs to create repeated tasks or schedule jobs to be performed later. CronJobs uses the Cron programming format used in Linux to program commands and scripts.
To learn more about CronJobs, read our
CronJob Kubernetes Guide. How to delete Kubernetes jobs Delete a job in Kubernetes using the following command: kubectl delete job [job-name] Alternatively, delete the job
using your YAML file:
kubectl delete -f [filename].yaml
The result confirms the successful deletion of the job
.
Conclusion
After reading this tutorial, you should know how to deploy and configure Kubernetes jobs to meet the needs of your cluster.
To learn more about Kubernetes objects, read How to create and use ConfigMap.