Introduction
I have had several requests to explain how I create my e-mail stats charts. This page describes how to set up *nix to record and generate e-mail statistics charts as used here. To get started, you need to make sure that you have exim, procmail and rrdtool all set up and working nicely.
The process involves maintaining a count of how many genuine and spam e-mails that are received. Procmail is responsible for filtering the e-mail and executing the appropriate script to increment the counters. A cron job runs periodically to generate the charts using rrdtool.
Exim Filter
The first thing to set up is an exim filter. The filter simply invokes
procmail:
# Exim filter <<== do not edit or remove this line! pipe "/usr/local/bin/procmail -t" seen finish
The filter is saved in your home directory (such as /home/jms/
)
as .forward
.procmailrc
The second file needed is the .procmailrc
file. This file
describes how e-mail is to be filtered by procmail. My .procmailrc
file looks like this:
:0 * ^X-UKC-SpamScore: sssssssssss { :0 c |/home/cug/jms30/maillog/eat-spam :0 Mail/spam } :0 * ^X-UKC-CSSpamCheck-Level: \*\*\*\*\*\*\* { :0 c |/home/cug/jms30/maillog/eat-spam :0 Mail/spam } :0 * ^X-UKC-CSSpamCheck-Flag: YES { :0 c |/home/cug/jms30/maillog/eat-spam :0 Mail/spam } :0 * ^X-UKC-SpamScore: ssssssssss { :0 c |/home/cug/jms30/maillog/eat-spam :0 Mail/spam } :0 * ^X-UKC-CSSpamCheck-Level: \*\*\*\* { :0 c |/home/cug/jms30/maillog/eat-spam :0 Mail/spam } :0 { :0 c |/home/cug/jms30/maillog/eat-inbox :0 .mail }
The first five sections match spam which is checked by my University. If it
matches any of the given headers, it executes the /home/cug/jms30/maillog/eat-spam
shell script (which increments the spam counter) and moves the e-mail to the
Mail/spam
directory. Anything that doesn't match as spam is
moved to my inbox (the last section).
You can modify your .procmailrc
file to match your requirements.
E-mail Counter Scripts
You will notice reference to two shell scripts in the .procmailrc
file - eat-spam
and eat-inbox
. Each of these scripts
simply executes another script called up-spam
or up-inbox
.
eat-spam
looks like this:
#! /bin/sh cat > /dev/null exec /home/cug/jms30/maillog/up-spam
and eat-inbox
looks like this:
#! /bin/sh cat > /dev/null exec /home/cug/jms30/maillog/up-inbox
up-spam
:
#! /bin/bash MAILLOGPATH=/home/cug/jms30/maillog printf '%d' $(($(cat $MAILLOGPATH/spam-counter) + 1)) > $MAILLOGPATH/spam-newcount mv $MAILLOGPATH/spam-newcount $MAILLOGPATH/spam-counter
and up-inbox
:
#! /bin/bash MAILLOGPATH=/home/cug/jms30/maillog printf '%d' $(($(cat $MAILLOGPATH/inbox-counter) + 1)) > $MAILLOGPATH/inbox-newcount mv $MAILLOGPATH/inbox-newcount $MAILLOGPATH/inbox-counter
You will need to change MAILLOGPATH
in both up-spam
and up-inbox
to point to the directory where your scripts are
stored. I keep mine in a directory called maillog
in my home
directory.
You will also need two files called spam-counter
and
inbox-counter
which actually keep the count of the number of spam
and genuine e-mails. Both these files will initially contain:
0
Finally you need to make sure that you can read and write to the counters and that you can execute the scripts.
rrdtool Setup
rrdtool is a tool for handling round-robin databases. A round-robin database is ideal for our requirements here because we are only charting the previous day's e-mail.
First off, we need to create the database. The database needs to contain two
data sources for the inbox and spam. Data is added every 15 minutes and an
archive of 96 readings are stored. The command needed to create the database
is:
rrdtool create data.rrd \ --step 900 \ DS:inbox:COUNTER:1800:U:U \ DS:spam:COUNTER:1800:U:U \ RRA:AVERAGE:0.5:1:96
Make sure that you don't overwrite any existing files - the database filename
will be data.rrd
.
Two scripts are periodically executed using a cron job. The first adds the
counter data to the round-robin database. This script is called
adddata
:
#! /bin/bash RRDTOOL=/usr/local/bin/rrdtool MAILLOGPATH=/home/cug/jms30/maillog for file in inbox spam; do cp $MAILLOGPATH/$file-counter $MAILLOGPATH/$file-curdata # printf "0" > $MAILLOGPATH/$file-counter done $RRDTOOL update $MAILLOGPATH/data.rrd \ -t inbox:spam \ N:$(cat $MAILLOGPATH/inbox-curdata):$(cat $MAILLOGPATH/spam-curdata) for file in inbox spam; do rm $MAILLOGPATH/$file-curdata done
You will need to change RRDTOOL
to be the path to rrdtool
and MAILLOGPATH
to be the path to your e-mail stats scripts.
The script to generate the charts is called graphdata
and
looks like this:
#! /bin/bash RRDTOOL=/usr/local/bin/rrdtool MAILLOGPATH=/home/cug/jms30/maillog GRAPHFILE=/home/cug/jms30/webpages/mailbox.png cd $MAILLOGPATH $RRDTOOL graph $GRAPHFILE \ --title="Daily mailbox traffic" \ DEF:inbox=data.rrd:inbox:AVERAGE \ CDEF:rinbox=inbox,900,\* \ DEF:spam=data.rrd:spam:AVERAGE \ CDEF:rspam=spam,900,\* \ 'AREA:rinbox#10c0d0:Inbox' \ 'GPRINT:rinbox:LAST:Current\: %8.0lf %s' \ 'GPRINT:rinbox:AVERAGE:Average\: %8.2lf %s' \ 'GPRINT:rinbox:MAX:Max\: %8.0lf %s\n' \ 'STACK:rspam#c04040:Spam' \ 'GPRINT:rspam:LAST: Current\: %8.0lf %s' \ 'GPRINT:rspam:AVERAGE:Average\: %8.2lf %s' \ 'GPRINT:rspam:MAX:Max\: %8.0lf %s' > /dev/null
Again, you will need to change RRDTOOL
to be the path to rrdtool,
MAILLOGPATH
to be the path to your e-mail stats scripts and
GRAPHFILE
tp be the path to where the charts should be saved
after they have been created.
Finally, you need a crontab to say when to execute the adddata
and graphdata
scripts:
0,15,30,45 * * * * /home/cug/jms30/maillog/adddata 1,16,31,46 * * * * /home/cug/jms30/maillog/graphdata
You can save your crontab in a file (say cron.tab
) and use
crontab cron.tab
to update your crontab.
This will make sure that the adddata
script is executed every
15 minutes and that the graphdata
script is executed every 15
minutes but one minute after the adddata
script.
© 2000-11, Jonathan Stott.
19 Apr 2024 02:49:42 GMT