Monit as PID 1 in a container

The first process in a container gets PID 1, and Linux treats PID 1 differently from other processes:

• Orphaned processes are re-parented to PID 1, which must reap them (collect their exit status) when they end. A process that is never reaped stays behind as a zombie and keeps its process ID, and when the IDs run out, no new process can be started.
• A signal sent to PID 1 is ignored unless PID 1 has a handler for it. A program without a SIGTERM handler ignores docker stop, and Docker kills it with SIGKILL 10 seconds later.

Many applications neither reap orphans nor handle SIGTERM. The usual fix is a small init such as tini or dumb-init, which starts the application as its child, reaps zombies and forwards signals to it. docker run --init adds tini to a container.

What Monit does as PID 1

Monit 5.35.0 and later takes care of both when it runs as PID 1:

• It reaps every child process that ends, including the orphans re-parented to it.
• On SIGTERM it stops all services with their stop programs, in reverse dependency order. Then it sends SIGTERM to every process still running, and SIGKILL to any that has not exited after 10 seconds.

More than an init

tini and dumb-init stop there. Monit also supervises the services in the container, just as it does on any server:

• It restarts a service when its process dies, when it stops answering on its port, or when it uses too much memory or CPU. If a service keeps failing, Monit can stop trying and alert you instead.
• It starts services in dependency order. When a service is restarted, the services that depend on it are stopped first and started again afterwards.
• It runs programs on a schedule, with cron syntax, and checks their exit status. Backups and cleanup jobs need no cron daemon in the container.
• It sends alerts by email, or runs a script of your choice, for example one that posts to a chat channel. With M/Monit you see the status of all your Monit instances in one place.
• It has a web interface and a command line for checking the services and for starting, stopping and restarting them.

A container that runs several services therefore needs no extra init, supervisor or cron daemon. Here is a monitrc for an application behind nginx, with a nightly backup:

set daemon 10
set mailserver smtp.example.com
set alert ops@example.com

check process myapp with pidfile /run/myapp.pid
    start program = "/usr/local/bin/myapp start"
    stop program  = "/usr/local/bin/myapp stop"
    if failed port 8080 protocol http request "/health" then restart
    if memory usage > 500 MB for 3 cycles then restart
    if 3 restarts within 5 cycles then unmonitor

check process nginx with pidfile /run/nginx.pid
    start program = "/usr/sbin/nginx"
    stop program  = "/usr/sbin/nginx -s quit"
    depends on myapp
    if failed port 80 protocol http then restart

check program backup with path "/usr/local/bin/backup.sh"
    every "0 3 * * *"
    if status != 0 then alert

Setup

Make Monit the entrypoint and run it in the foreground with -I (or set init in monitrc):

COPY monitrc /etc/monitrc
RUN chmod 600 /etc/monitrc
ENTRYPOINT ["/usr/local/bin/monit", "-I", "-c", "/etc/monitrc"]

Don't add docker run --init: tini is then PID 1, and on shutdown the services are killed instead of stopped.

Docker kills a container that has not stopped within its stop timeout, 10 seconds by default. If your services need longer to stop, set the timeout explicitly with docker stop -t, docker run --stop-timeout or stop_grace_period in Docker Compose.

Try it

The monit-docker-test repository on GitHub sets up a test container with Monit as PID 1. Its README goes step by step through checks of zombie reaping, service restarts and shutdown.