Wednesday, October 26, 2022

[SOLVED] kubernetes cronjob unexpected scheduling behavior

Issue

I'm using kubernetes 1.21 cronjob to schedule a few jobs to run at a certain time every day.

I scheduled a job to be run at 4pm, via kubectl apply -f <name of yaml file>. Subsequently, I updated the yaml schedule: "0 22 * * *" to trigger the job at 10pm, using the same command kubectl apply -f <name of yaml file> However, after applying the configuration at around 1pm, the job still triggers at 4pm (shouldn't have happened), and then triggers again at 10pm (intended trigger time).

Is there an explanation as to why this happens, and can I prevent it?

Sample yaml for the cronjob below:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: job-name-1
spec:
  schedule: "0 16 * * *" # 4pm
  successfulJobsHistoryLimit: 1
  failedJobsHistoryLimit: 1
  jobTemplate:
    spec:
      template:
        spec:
          containers:
            - image: sample-image
              name: job-name-1
              args:
                - node
                - ./built/script.js
              env:
                - name: NODE_OPTIONS
                  value: "--max-old-space-size=5000"
          restartPolicy: Never
          nodeSelector:
            app: cronjob

I'm expecting the job to only trigger at 10pm.

Delete the cronjob and reapply it seems to eliminate such issues, but there are scenarios where I cannot the delete the job (because it's still running).


Solution

  1. As you use kubectl apply -f <name of yaml file> to schedule a second Job at 10pm which means it will schedule a new Job but it will not replace the existing job. so the reason was that the Job at 4pm also scheduled and it runned.

Instead you need to use the below command to replace the Job with another scheduled Job.

kubectl patch cronjob my-cronjob -p '{"spec":{"schedule": "0 22 * * *"}}'

This will run Job only at 10Pm.

  1. In order to delete the running Job use the below Process :

run in console:

crontab -e

then you will get crontab opened with an editor, simply delete the line there, save the file and quit the editor - that's it.

if you are running with a root user then use the below command and proceed as above step.

sudo crontab -e


Answered By - Hemanth Kumar
Answer Checked By - Mildred Charles (WPSolving Admin)
at