How To Schedule Python Scripts As Cron Jobs With Crontab (Mac/Linux)

Learn about cron jobs and how to schedule commands and Python scripts in the terminal via crontab (for Linux and Mac)

#more

In this tutorial, we learn about cron jobs and how to schedule commands and Python scripts in the terminal via crontab (for Linux and Mac). This allows us to run commands on a repetitive schedule. We specifically look into running Python scripts as cron jobs. There are a couple of pitfalls where we have to be careful. We also learn how to schedule jobs with a virtual environment.

The code is also available on GitHub.

Crontab¶

We can schedule jobs with the crontab command. A crontab file is a simple text file containing a list of commands meant to be run at specified times. It is edited using the crontab command. The commands in the crontab file are checked by the cron daemon, which executes them in the system background.

List all scheduled cron jobs with:

$ crontab -l

Edit the file by using the command:

$ crontab -e

Then use the following syntax to schedule a job

* * * * * command

For example to run a python script use:

* * * * * python my_script.py

The syntax¶

To check the syntax I recommend to visit the free website crontab.guru.

The syntax is this: 'minute hour day-of-month month day-of-week'

The 5 items are used to specify minute, hour, day in the month, month, and day in the week (in this order).

Allowed values are:

• 0-59 for minute
• 0-23 for hour
• 1-31 for day-of-month
• 1-12 for month
• 0-6 for day-of-week (0 = Sunday)

The following characters can be used for more complex timing:

• * any value
• , value list separator
• - range of values
• / step values

Some examples:

• * * * * *: At every minute (every minute, every hour, every day, every month, every weekday)
• 0 10 * * *: At 10:00 AM each day
• 0 10 * * 0: At 10:00 AM on each Monday
• 0 0 1,15 * * At 00:00 on 1st and 15th each month
• 0 0 * * 1-3: At 00:00 from Monday through Wednesday
• 0 0 1/2 * *: At 00:00 on every 2nd day from 1 through 31

Pitfalls when running Python scripts¶

\1. Use the full path to your file, e.g.:

* * * * * python ~/code/python-task-automation/test.py

\2. When logging into files, specify the full path for the filename in the Python file. Otherwise the file just ends up in your home directory.

import logging
import os

dir_path = os.path.dirname(os.path.realpath(__file__))
filename = os.path.join(dir_path, 'test_log.log')

# Logger
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)

file_handler = logging.FileHandler(filename)
file_handler.setLevel(logging.INFO)
file_handler.setFormatter(logging.Formatter('%(asctime)s - %(levelname)s - %(message)s'))
logger.addHandler(file_handler)

\3. When a virtual environment is used, specify the full path to the python executable, e.g.:

* * * * * ~/opt/miniconda3/envs/py38/bin/python ~/code/python-task-automation/fetch_github.py
or
* * * * * ~/code/python-task-automation/venv/bin/python ~/code/python-task-automation/fetch_github.py