Scheduling background tasks using cron in a Docker container

By: (plus.google.com) +David Herron; Date: May 27, 2018

Tags: Docker » Docker MAMP

Sometimes you want a Docker container to execute background tasks, and therefore want cron to be installed and running. Having cron running in the background is part of normal Unix/Linux/etc system admin practices. Even though the crontab format is kind of hokey, we all learn it and set up automated background tasks to keep the world functioning. Let's see how to set this up in a Docker container.

My first thought to ever run a cron service in Docker came while writing the 4th Edition of Node.js Web Development. The book covers the full gamut of Node.js application development from soup to nuts, that is from initial concept to delivery on real cloud hosting servers, and even consideration of security setup. The last included setting up HTTPS service for a Node.js application, since the web is moving strongly to requiring HTTPS on every website.

I wanted to demonstrate setting up Let's Encrypt in a Docker container. Let's Encrypt is a $0 cost free service for getting the SSL certificates required to run an HTTPS server. Since I didn't find any tutorials on running Let's Encrypt in a Docker container, I had to invent something. The tooling to run Let's Encrypt involves running a command-line program every day, and that program checks for any SSL certificates which need to be renewed. Having an occasionally executed background tasks simply screams cron.

Well, that's true for someone like myself who learned Unix administration in the 1980's. The youngsters taking over the world may have a better idea sooner-or-later and replace cron with something else.

There's approximately three scenarios to consider:

  1. A single-purpose container containing just a cron daemon
  2. A container with two services, the cron daemon and another
  3. A container with several services, a process manager, and having cron as one of the services

Docker container running JUST a cron daemon

In Node.js Web Development, I showed this Dockerfile:

FROM debian:jessie

# Install cron, certbot, bash, plus any other dependencies
# (1) install cron
RUN apt-get update && apt-get install -y cron bash wget
RUN mkdir -p /webroots/evsoul.com/.well-known && mkdir -p /scripts
WORKDIR /scripts
RUN wget https://dl.eff.org/certbot-auto
RUN chmod a+x ./certbot-auto
# Run certbot-auto so that it installs itself
RUN /scripts/certbot-auto -n certificates
# /webroots/DOMAIN.TLD/.well-known/... files go here
VOLUME /webroots
VOLUME /etc/letsencrypt

# (2) Set up a crontab entry
# This installs a Crontab entry which
# runs "certbot renew" on the 2nd and 7th day of each week at 03:22 AM
#
# cron(8) says the Debian cron daemon reads the files in /etc/cron.d,
# merging into the data from /etc/crontab, to use as the system-wide cron jobs
#
# RUN echo "22 03 * * 2,7 root /scripts/certbot-auto renew" >/etc/cron.d/certbot

# (3) Start cron in the foreground
CMD [ "cron", "-f" ]

There's some Let's Encrypt specifics in this, so let's focus on a couple key points.

  1. Make sure to install the cron daemon, using the method for your preferred OS. In this case we're running Debian Jessie, and therefore use apt-get install cron
  2. Set up the required crontab entries. The example shown here, creating a file in /etc/cron.d, is correct for the cron implementiation in Debin Jessie. It seemed from my research that different OS's have differences in how to do this. This method can be performed when the container is built, since it's just a file plopped into the file system.
  3. Start cron in the foreground using the -f flag.

The last is important both here and in the next section. Docker considers the container is successfully running as long as the process started in the CMD instruction is still in existence. As soon as that process exits, Docker thinks the container has died. Depending on the setting of the restart option, Docker will either kill the container, or restart it, if it believes the container has died.

By running cron -f the cron daemon stays in the foreground, does not spin itself into the background, and therefore Docker knows the daemon is still running, and thinks the container is still alive.

In this case we only need one crontab entry. Your application may require multiple crontab entries. It's simple enough to replicate this RUN command enough times to create all the entries you require.

Docker container with both cron and another service

A couple months ago I wanted to use Let's Encrypt to provision SSL certificates for services I run at home and expose to the world. For details, see Easily use Let's Encrypt to HTTPS-protect your own server, for free

In that case, I wanted to run both nginx and cron in the same container. It's important to decide between a standalone cron container, versus bundling cron into a container with one or more services.

In Node.js Web Development, I rationalized the standalone cron as the necessity to provision SSL certificates in one place. The book describes building a server application where one may want to deploy 10's or 100's of instances for load balancing or whatever other purpose. Setting up Let's Encrypt SSL management in every such container doesn't make sense in the slightest, because SSL provisioning needs to be centralized.

For my home server, I wanted to use nginx to proxy several servers and to use nginx to implement the SSL. There would be one nginx service, so therefore it would be great to have the Let's Encrypt cron jobs running inside that same container.

Bottom line - for some applications you need a standalone cron service, and in other applications the cron service needs to be integrated with another service. You might even have cases where cron services exist in multiple containers.

Here's the implementation:

FROM nginx:stable

# Inspiration:
# https://hub.docker.com/r/gaafar/cron/

# (1) Install cron, certbot, bash, plus any other dependencies
RUN apt-get update \
   && apt-get install -y cron bash wget
RUN mkdir -p /webroots/ /scripts

# /webroots/DOMAIN.TLD/.well-known/... files go here
VOLUME /webroots
VOLUME /etc/letsencrypt

# /webroots/DOMAIN.TLD will be mounted 
# into each proxy as http://DOMAIN.TLD/.well-known
#
# /scripts will contain certbot and other scripts

COPY register /scripts/
RUN chmod +x /scripts/register

WORKDIR /scripts
RUN wget https://dl.eff.org/certbot-auto
RUN chmod a+x ./certbot-auto
# Run certbot-auto so that it installs itself
RUN /scripts/certbot-auto -n certificates

# (2) This installs a Crontab entry which 
# runs "certbot renew" on several days a week at 03:22 AM
#
RUN echo "22 03 * * 2,4,6,7 root /scripts/certbot-auto renew" >/etc/cron.d/certbot

# (3) Run both nginx and cron together
CMD [ "sh", "-c", "nginx && cron -f" ]

The three steps are pretty much the same as before. There is a difference with the CMD instruction, since we need to start two services.

I experimented a lot with this until developing this particular invocation.

In the official nginx Docker container, the CMD instruction is a little different (see (github.com) https://github.com/nginxinc/docker-nginx/blob/590f9ba27d6d11da346440682891bee6694245f5/mainline/stretch/Dockerfile)

CMD ["nginx", "-g", "daemon off;"]

The -g option sets global configuration settings. Setting daemon off means that nginx will not spin itself into the background, and instead stay in the foreground. (see (nginx.org) https://nginx.org/en/docs/switches.html)

In order to start two (or more) commands in one command-line, we need the first few to spin themselves into the background. The only command which should stay in the foreground must be the final command in the sequence. Hence we need nginx to spin into the background, and for cron to stay in the foreground.

Or, do we need cron in the background and nginx to stay in the foreground? If you believe that to be the case, the two should be reversed. I wasn't able to get that combination to work correctly, however. I had a lot of difficulty getting a CMD instruction to launch two processes in any syntax other than what's shown here.

Docker container with multiple services, including cron

A little further down the slippery slope is those Docker containers consisting of several service processes in one container. Two that I'm familiar with are Gitlab and Gogs, both of which are Github alternatives. I'm using Gogs in the system I mentioned in the previous section.

The official Gogs Dockerfile uses Alpine Linux and something called s6-svcscan to start up multiple services. (See (github.com) https://github.com/gogs/gogs/blob/master/Dockerfile)

The s6-svscan program scans for multiple s6-supervice services to manage, and it starts all of them. S6 is a suite of programs to manage process supervision.

This is only one example of a process supervision system. One could plausibly run traditional /etc/init in a Docker container as process #1 just like you'd do on a regular Linux/Unix system. Or you could use any of the other process supervisors.

I don't have an example to show in this case, other than to refer you to the Gogs Dockerfile.

« DIY Solar powered Weather Station [WiFi, MQTT, Smart Home, ESP8266] Introduction to Node.js with the Serverless framework on AWS Lambda »
2016 Election Acer C720 Ad block AkashaCMS Amazon Amazon Kindle Amazon Web Services America Amiga and Jon Pertwee Android Anti-Fascism AntiVirus Software Apple Apple Hardware History Apple iPhone Apple iPhone Hardware April 1st Arduino ARM Compilation Artificial Intelligence Astronomy Astrophotography Asynchronous Programming Authoritarianism Automated Social Posting AWS DynamoDB AWS Lambda Ayo.JS Bells Law Big Brother Big Finish Bitcoin Mining Black Holes Blade Runner Blockchain Blogger Blogging Books Botnets Cassette Tapes Cellphones China China Manufacturing Christopher Eccleston Chrome Chrome Apps Chromebook Chromebox ChromeOS CIA CitiCards Citizen Journalism Civil Liberties Clinton Cluster Computing Command Line Tools Comment Systems Computer Accessories Computer Hardware Computer Repair Computers Cross Compilation Crouton Cryptocurrency Curiosity Rover Currencies Cyber Security Cybermen Daleks Darth Vader Data backup Data Storage Database Database Backup Databases David Tenant DDoS Botnet Detect Adblocker Developers Editors Digital Photography Diskless Booting Disqus DIY DIY Repair DNP3 Do it yourself Docker Docker MAMP Docker Swarm Doctor Who Doctor Who Paradox Doctor Who Review Drobo Drupal Drupal Themes DVD E-Books E-Readers Early Computers Election Hacks Electric Bicycles Electric Vehicles Electron Emdebian Encabulators Energy Efficiency Enterprise Node EPUB ESP8266 Ethical Curation Eurovision Event Driven Asynchronous Express Face Recognition Facebook Fake News Fedora VirtualBox File transfer without iTunes FireFly Flickr Fraud Freedom of Speech Front-end Development Gallifrey git Github GitKraken Gitlab GMAIL Google Google Chrome Google Gnome Google+ Government Spying Great Britain Heat Loss Hibernate Hoax Science Home Automation HTTP Security HTTPS Human ID I2C Protocol Image Analysis Image Conversion Image Processing ImageMagick In-memory Computing InfluxDB Infrared Thermometers Insulation Internet Internet Advertising Internet Law Internet of Things Internet Policy Internet Privacy iOS Devices iPad iPhone iPhone hacking Iron Man iTunes Java JavaScript JavaScript Injection JDBC John Simms Journalism Joyent Kaspersky Labs Kindle Kindle Marketplace Lets Encrypt LibreOffice Linux Linux Hints Linux Single Board Computers Logging Mac Mini Mac OS Mac OS X Machine Learning Machine Readable ID macOS MacOS X setup Make Money Online March For Our Lives MariaDB Mars Mass Violence Matt Lucas MEADS Anti-Missile Mercurial MERN Stack Michele Gomez Micro Apartments Microsoft Military AI Military Hardware Minification Minimized CSS Minimized HTML Minimized JavaScript Missy Mobile Applications Mobile Computers MODBUS Mondas Monetary System MongoDB Mongoose Monty Python MQTT Music Player Music Streaming MySQL NanoPi Nardole NASA Net Neutrality Network Attached Storage Node Web Development Node.js Node.js Database Node.js Testing Node.JS Web Development Node.x North Korea npm NVIDIA NY Times Online advertising Online Community Online Fraud Online Journalism Online Photography Online Video Open Media Vault Open Source Open Source Governance Open Source Licenses Open Source Software OpenAPI OpenVPN Palmtop PDA Patrick Troughton Paywalls Personal Flight Peter Capaldi Phishing Photography PHP Plex Plex Media Server Political Protest Postal Service Power Control Privacy Production use Public Violence Raspberry Pi Raspberry Pi 3 Raspberry Pi Zero ReactJS Recaptcha Recycling Refurbished Computers Remote Desktop Removable Storage Republicans Retro Computing Retro-Technology Reviews RFID Right to Repair River Song Robotics Rocket Ships RSS News Readers rsync Russia Russia Troll Factory Russian Hacking Rust SCADA Scheme Science Fiction SD Cards Search Engine Ranking Season 1 Season 10 Season 11 Security Security Cameras Server-side JavaScript Serverless Framework Servers Shell Scripts Silence Simsimi Skype SmugMug Social Media Social Media Warfare Social Network Management Social Networks Software Development Space Flight Space Ship Reuse Space Ships SpaceX Spear Phishing Spring Spring Boot Spy Satellites SQLite3 SSD Drives SSD upgrade SSH SSH Key SSL Stand For Truth Strange Parts Swagger Synchronizing Files Telescopes Terrorism The Cybermen The Daleks The Master Time-Series Database Tom Baker Torchwood Total Information Awareness Trump Trump Administration Trump Campaign Twitter Ubuntu Udemy UDOO US Department of Defense Virtual Private Networks VirtualBox VLC VNC VOIP Vue.js Web Applications Web Developer Resources Web Development Web Development Tools Web Marketing Webpack Website Advertising Weeping Angels WhatsApp William Hartnell Window Insulation Windows Windows Alternatives Wordpress World Wide Web Yahoo YouTube YouTube Monetization