How to write a shell script restart apache if `server reached MaxClients`

Question

Recently, my blog always shutdown, it use centos6.2+apache2.2+mysql5.5+php5.3. I have raised MaxClients in httpd.conf, but less useful.

So is it possible to write a shell script (Run with crontab every 10 minutes), to read the httpd/error_log, if the last message preg_match sending a SIGTERM do restart apache automatically?

Answer 1

Parsing the log files can be pretty tricky. Rather than try to do that, you'd probably be better off using a script such as this one which can run from a crontab entry. This script will attempt to access the server, if it's unsuccessful then it will restart Apache.

Script

Source of script: bash script to restart Apache automatically

#!/bin/sh
# Script that checks whether apache is still up, and if not:
# - e-mail the last bit of log files
# - kick some life back into it
# -- Thomas, 20050606

PATH=/bin:/usr/bin
THEDIR=/tmp/apache-watchdog
[email protected]
mkdir -p $THEDIR

if ( wget --timeout=30 -q -P $THEDIR http://localhost/robots.txt )
then
    # we are up
    touch ~/.apache-was-up
else
    # down! but if it was down already, don't keep spamming
    if [[ -f ~/.apache-was-up ]]
    then
        # write a nice e-mail
        echo -n "apache crashed at " > $THEDIR/mail
        date >> $THEDIR/mail
        echo >> $THEDIR/mail
        echo "Access log:" >> $THEDIR/mail
        tail -n 30 /var/log/apache2_access/current >> $THEDIR/mail
        echo >> $THEDIR/mail
        echo "Error log:" >> $THEDIR/mail
        tail -n 30 /var/log/apache2_error/current >> $THEDIR/mail
        echo >> $THEDIR/mail
        # kick apache
        echo "Now kicking apache..." >> $THEDIR/mail
        /etc/init.d/apache2 stop >> $THEDIR/mail 2>&1
        killall -9 apache2 >> $THEDIR/mail 2>&1
        /etc/init.d/apache2 start >> $THEDIR/mail 2>&1
        # send the mail
        echo >> $THEDIR/mail
        echo "Good luck troubleshooting!" >> $THEDIR/mail
        mail -s "apache-watchdog: apache crashed" $EMAIL < $THEDIR/mail
        rm ~/.apache-was-up
    fi
fi

rm -rf $THEDIR

Paths

The paths to the stop/start scripts will need to be adjusted accordingly based on where your distro has installed Apache. Lines like this one:

        /etc/init.d/apache2 start >> $THEDIR/mail 2>&1

If you're on CentOS will be like this:

        /etc/init.d/httpd start >> $THEDIR/mail 2>&1

Name of Executable

The same goes with the killlall lines. The name of the executable on CentOS is httpd.

The crontab entry

This cron will need to be run as root so that it has the appropriate permissions to stop/start Apache.