Schedule and grace
When exactly we decide a job did not run.
The monitor waits for a ping until the expected time plus the grace period. If none arrives, it opens an incident and sends an alert.
Choosing the grace period
Grace is not “how long the job takes” but “how late it may start”. A nightly backup that starts sharply at three is fine with five minutes. A job waiting in a shared runner queue needs an hour.
Too tight is worse than too loose. False alarms train people to ignore notifications — and the real outage gets lost among them.
Maximum duration
If the job sends a start signal, we also watch how long it runs. The deadline is the earlier of two points: “expected time plus grace” and “start plus maximum duration”. A hung job is caught without waiting for the next scheduled run.
Timezones and DST
The schedule is evaluated in the monitor timezone, DST included. On a clock-change day a “daily at 03:00” job is expected once — not twice, not zero times — matching how system cron behaves.