I find it kind of tragic that System V (SysV) init (/etc/init.d, etc) has survived as long as it has. Looking at its features, it's a pretty plain system, so why is it still here? Maybe because, historically, it has been good enough. However, in the places it hasn't been good enough, folks made workarounds, extensions, and even efforts to vanquish it entirely.
Sometimes these workarounds and extensions are done improperly. SysV init often forces upon programs common tasks such as pid file management, daemonizing (backgrounding), log management, privilege separation, etc. Most things don't use pid files correctly (nobody locks them), many startup scripts lack a functioning 'status' check, and everyone reimplements the daemonizing and privilege dropping differently and many times incorrectly. Yuck, I want less of that.
What more is needed? Automatic restarts, centralized control, common configuration, etc.
What if you need more? That's where supervisord comes in. Supervisord gets you a few more goodies you may find useful, including a management web interface, an API, including process startup backoff.
Installation is pretty simple and comes with a sample config generator for helping decrease the learning curve. Further, the config format looks like INI format, so the learning curve there should also be pretty short.
I decided to try putting mysql into supervisord for testing, so after creating the default config file:
# echo_supervisord_conf > /etc/supervisord.confI checked ps(1) for how I was running mysql, and put that invocation in
[program:mysqld] command=/usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --port=3306 --socket=/var/run/mysqld/mysqld.sockThen I ran supervisord (you probably want supervisord to to have an init script, so it launches on boot, etc)
% sudo supervisord # it's running, so let's check on our mysql server: snack(~) % sudo supervisorctl status mysqld RUNNING pid 26028, uptime 0:00:28Hurray! Supervisord even tries understanding what exit statuses mean. If I send SIGTERM to mysqld, then mysqld will shutdown gracefully and exit with status 0. By default, supervisord is configured that an exitcode of 0 is "expected" and thus it won't restart the process. You can change this by setting the 'autorestart' option to 'true' or by changing which exit codes supervisord understands as expected with the 'exitcodes' option:
[program:mysqld] command=/usr/sbin/mysqld ... autorestart=trueSupervisord can be told to reload (restarts itself) or to reread the config file. In this case, we can just tell it to reread the config file:
snack(~) % sudo supervisorctl reread mysqld: changedNow any mysqld exit will be restarted automatically.
Additional features include the management web interface, which can be enabled
in the config file with the
configuration and also with the
configuration for local-only management. The supervisorctl tool can talk to
remote servers, so the http server portion isn't just for people.
Supervisord also seems easy to extend and supports event notifications if you need it. Supervisord also handles logging of stdout and stderr for you in a reasonably configurable way.
At time of writing, there are a few shortfalls.
- No HTTPS support on the management interface
- No decent per-program access control
- No 'retry forever' option.
startretriesdefaults to 3 and has no value for infinity. Perhaps setting a really huge value for startretries is a reasonable workaround. I tried
startretries=100000000which seems to work,
Supervisord may not replace all of your startup scripts, but I highly recommend it, or something like it, for your important services.