[unclug] ACK! Spam!

Alan Hoyle alanh at email.unc.edu
Wed Feb 18 13:42:12 EST 2004

I get lots of spam.  Since the beginning of Feb, I've received almost
10000 spams.  Yes, that's ten thousand.  My setup has caught and deleted
9668 of them at last count (True positive).  It didn't catch roughly 260
for a false negative rate of around 3%.  Since the beginning of the year,
classified as spam roughly 5 non-spam messages (false positive).  It's
hard to tell given the sheer volume of spams that I get. I don't know how
many "true negatives" I've had (i.e. non-spams classified as non-spams).

Pine's "Rules" setup doesn't really work well.  I tried that for a long

Here's what I did on my linux desktop.  I don't know if it would work on
isis.  I run the pine out of AFS package space.

1.  Downloaded and installed IMAPSpamBegone

2.  Installed SpamAssassin (with RedHat9)

3.  Noticed that SA was an absolute memory and processor pig on my
    desktop machine.

4.  Downloaded and installed Bogofilter (bogofilter.sf.net)

   4a.  trained bogofilter with collection of spam and non-spam messages

5.  Modified ISBG (it's open-source Python) so it works with bogofilter
    (in addition to some other changes)

6.  Installed isbg into my crontab so it runs every 20 seconds or so
    during the day and every 5 minutes or so at night.  Used the pine
    pipe command ('|') to train on messages in my inbox.  (bogofilter -Ns
    is the most common command I use)

... time passes ...

7.  Noticed that every once in a while, ISBG hangs on certain messages
    (particularly some weird virus messages) or when the mailserver or
    network goes down.  the subsequent build-up of isbg processes
    eventually crashes my machine if unchecked.

8.  Added a "killall isbg" every 5 min to my crontab.

... time passes ...

9.  Upgraded bogofilter.

10.  Noticed how it broke my ISBG setup.

11.  wrote "deisbg" script to remove it from my crontab temporarily and
     add it back when I'm done.

... time passes ...

12.  got tired of typing "bogofilter" in the | dialog box, created "bf"
     shell script to run it.

13.  Noticed that the .bogofilter word list gets big fast.

14.  Automated the "compact DB" function to run weekly in my crontab.
     (weekly run, reduces size from ~100 MB to ~65 MB, yes, it grows
     almost 40 MB/week)

If anyone's interested, I can post more details on my setup.

On Wed, 18 Feb 2004 at 12:09, Cristobal M Palmer wrote:

> I've gotten used to using the isis-based Pine program for my UNC-CH email
> account, but recently I've started getting a lot more spam.
> UNC has server-side filtering, but I can't figure out if there's a
> user .config that I could use to train Pine to recognize junk. I know
> there's a 'rules' option menu, but that looks like a lot of work.
> If that'd just be too much of a hassle, can anyone recommend a
> **lightweight** email client with Bayesian filtering?
> I want to hit a 'delete as junk' button and not have to see the same
> message again.
> Sick of hearing about NEW!!1 weight-|oss pi1ls...

    Alan Hoyle  -  alanh at unc.edu  -  http://www.alanhoyle.com/
      "I don't want the world, I just want your half." -TMBG
                 Get Horizontal, Play Ultimate.

More information about the unclug mailing list