Know when a cron job, backup or scheduled task silently stops

How it works

Website and server monitors check from the outside. A job that runs on its own schedule has nothing to visit, so a heartbeat works the other way around: the job calls Monitorizer.

Set one up in three steps

1. 1

   Add a heartbeat monitor

   On the website: Monitors, Add monitor, Heartbeat. In the Mac app: Add, Heartbeat. Give it a clear name, such as "Nightly backup, shop server". Heartbeats need your account, since the calls arrive on the website.

2. 2

   Paste your crontab line

   Paste the job's line from crontab -e: the schedule, a name and a grace time are filled in for you. No crontab line (Windows, GitHub Actions)? Set the interval or schedule by hand. Check the time zone: the one of the machine that runs the job.

3. 3

   Copy the line back

   Save. The monitor's page gives your line back with the ping added, in a simple version or one that also reports failures. Replace the old line with it, and the first ping appears within seconds of the next run.

Interval or cron schedule?

A cron schedule has five fields, separated by spaces:

┌───────── minute (0 to 59)
│ ┌─────── hour (0 to 23)
│ │ ┌───── day of the month (1 to 31, L = last)
│ │ │ ┌─── month (1 to 12)
│ │ │ │ ┌─ day of the week (0 to 6, Sunday = 0)
│ │ │ │ │
0 3 1 * *

Choose the time zone of the machine that runs the job, so "03:00" means the same thing on both sides. Daylight saving time is handled for you.

Choosing the grace time

The grace time is how long after the expected time a call may still arrive. It covers the time the job takes and small delays. Too short gives false alarms, too long delays real ones.

• A quick job (a few seconds) every 5 to 15 minutes: 5 minutes.
• An hourly job that takes a few minutes: 15 to 30 minutes.
• A nightly backup that takes 20 minutes: 1 hour.
• A weekly or monthly job that can take hours: 2 to 6 hours.
• Rule of thumb: the longest the job has ever taken, plus a margin.

The ping address

GET, POST and HEAD requests all work, so any tool that can open a web address can ping.

Keep the address private, like a password: anyone who has it can send calls for this monitor. If it leaks, delete the monitor and create a new one, which gets a new address.

Examples

Replace YOUR-ADDRESS with the address shown on your monitor's page (it has copy buttons).

# The call is made only if the backup succeeded (&&).
0 3 * * * /home/me/backup.sh && curl -fsS -m 10 --retry 3 -o /dev/null https://monitorizer.ai.tn/hb/YOUR-ADDRESS

#!/bin/sh
# Reports the start, the end, and a failure with the end of the log.
HB=https://monitorizer.ai.tn/hb/YOUR-ADDRESS
curl -fsS -m 10 -o /dev/null $HB/start
if /home/me/backup.sh > /tmp/backup.log 2>&1; then
  curl -fsS -m 10 -o /dev/null $HB
else
  tail -c 1000 /tmp/backup.log | curl -fsS -m 10 -o /dev/null --data-binary @- $HB/fail
fi

# /etc/systemd/system/backup.service
[Service]
Type=oneshot
ExecStart=/home/me/backup.sh
ExecStartPost=/usr/bin/curl -fsS -m 10 -o /dev/null https://monitorizer.ai.tn/hb/YOUR-ADDRESS

# Run this script from the scheduled task.
& "C:\Scripts\backup.ps1"
if ($?) { Invoke-RestMethod https://monitorizer.ai.tn/hb/YOUR-ADDRESS }
else { Invoke-RestMethod -Method Post -Body "Backup failed" https://monitorizer.ai.tn/hb/YOUR-ADDRESS/fail }

# .github/workflows/nightly.yml
on:
  schedule:
    - cron: "0 3 * * *"
jobs:
  nightly:
    runs-on: ubuntu-latest
    steps:
      - run: ./nightly.sh
      - run: curl -fsS -m 10 ${{ secrets.HEARTBEAT_URL }}
      - if: failure()
        run: curl -fsS -m 10 ${{ secrets.HEARTBEAT_URL }}/fail

<?php
// At the end of the scheduled task.
run_my_task();
file_get_contents('https://monitorizer.ai.tn/hb/YOUR-ADDRESS');
// WordPress: wp_remote_get('https://monitorizer.ai.tn/hb/YOUR-ADDRESS');

try {
  await runJob();
  await fetch("https://monitorizer.ai.tn/hb/YOUR-ADDRESS");
} catch (error) {
  await fetch("https://monitorizer.ai.tn/hb/YOUR-ADDRESS/fail", { method: "POST", body: String(error) });
}

import urllib.request

try:
    run_job()
    urllib.request.urlopen("https://monitorizer.ai.tn/hb/YOUR-ADDRESS", timeout=10)
except Exception as error:
    urllib.request.urlopen("https://monitorizer.ai.tn/hb/YOUR-ADDRESS/fail", data=str(error).encode(), timeout=10)

Questions