Sysadmins, at the switch to daylight saving time: have you thought about your cron jobs?

The good news is that if you don’t know what a crontab or a cron job is, this doesn’t concern you, so you can go about your business.  Everyone else, read on, it may concern you.

At every switch from standard time to daylight saving time, or the other way round, there’s food for thought regarding the cron jobs on your Unix and Linux servers, and the scheduled tasks on your Windows servers.  There are several possible scenarios, and one precaution to take: move your scheduled jobs out of the sensitive slot [2:00 - 2:59am].

Machines that implement the switch

This is obviously the vast majority of production servers in the parts of the world where the locale settings require such a switch, i.e.  mainly the countries and territories located beyond the tropics, whatever the hemisphere; careful, there are exceptions.

For the rest of this article, I assume we’re talking about a server subject to a time change.  For the others, there’s nothing to plan.

Switching to daylight saving time

In France, it’s at 2am on the last Sunday of March that the clocks instantly go forward an hour, so the [2:00-2:59am] slot is never lived that day, neither by humans nor by machines programmed with the switch.  This means any job scheduled in that slot won’t run unless your operating system version implements automatic rescheduling of scheduled tasks (e.g.  OpenServer).
If, after checking the official documentation, it turns out the OS doesn’t implement automatic job rescheduling, and if scheduled jobs are chained in a sequence where later jobs depend on earlier ones in any way, such as the presence of certain data, or of certain files created by the previous jobs, you’ll need to shift them forward or backward by a fixed amount, so that no job starts in that slot, which doesn’t exist as far as machine time goes.  If you’re not sure how the OS behaves, or don’t control their run time, I’d advise shifting them forward, i.e.  from 3:00am.

The edge case of 2:00am

By default, the assumption is that 2:00am will trigger no job, because the clock will be changed before the events tied to it occur.  When in doubt, apply this assumption as a precaution.

Switching back to standard time

Unsurprisingly, the opposite effect lies in wait for you.  At 3am, the system clock will go back to 2am.  So any jobs scheduled in that slot would run twice, or at least be started twice, and depending on their implementation, could play out their scenario twice, with all the consequences, foreseeable or not, that would follow.

The edge case of 3:00am

Just as for the reverse switch, the one to daylight saving time, the default assumption is that 3:00am will trigger no job the first time, because the clock will be changed before the events tied to it occur; but the second time, i.e.  one hour later, after the clock goes back an hour, 3:00am will indeed run.  So in the end it should only run once.

Machines that don’t implement the switch

These days it’s rare for production machines not to implement the switch; however, if it happens, there’s nothing special to plan, since by early Sunday morning the sensitive slot has been lived, and any tasks executed.  It’s up to you, usually on Sunday morning or during the day, to update the system clock one way or the other. If you’re up and in front of a computer at switch time, I’m truly sorry, but it happens; so whatever you do, don’t change the system time at 2:00 or 3:00am before making sure no job is going to run twice, or on the contrary be skipped.

Conclusion

If you’re the administrator of a production machine, or of a machine on which running scheduled tasks matters, then, whatever the operating system and its scheduling system, you should review the task schedule and spot those set to run on the Sunday of a time change, in the [2:00-2:59am] slot.  Once identified, ask yourself what the effect on your system would be if they didn’t run that day, or ran twice that day.  In particular, are they part of a sequence of tasks where each depends on the previous one? If you’re not sure, reschedule them outside this sensitive period.