Security

The Problem with Spam
by Will McCullen

I hate e-mail spam!  I mean I really hate it.  It comes in like the most annoying commercial, the most base and vulgar intrusion or the most subtle of traps.  It wastes time, offends people and wastes money.  On top of that it is like fighting the hydra.  Once you think you kill one source, three more pop up in its place.  It is a pain.  Whenever I bear down on the spam inevitably some legitimate e-mail gets lost.  If I let up, then some of you poor souls get lambasted.  Grrrrrr!  I really hate e-mail spam!  (have I said that?)

All the e-mail that comes into the district first hits what we affectionately call the spambox.  This is a server that checks incoming messages against a list of known attributes, signatures if you will, that positively identify a message as spam.  This list is updated multiple times a day and is created from a company with listening servers all over the globe.  When a message is checked against this list and is identified as spam, we can be very confident that it is indeed spam. 

This server will also check e-mail attributes that it suspects might be spam.  This type of checking is a little different.  It is more of a formula which looks for common elements that regularly occur in spam.  Each attribute is given a score.  When an e-mail is given a score that is high enough to meet the threshold we set, it is then counted as spam.  The server defines this as 'suspected spam'.  Great...so what does that really mean, Will?

Glad you asked...;-)

The server sees incoming mail as either definite spam, suspected spam or legit e-mail.  The part we can adjust is the threshold between suspected spam and legit e-mail.  If we adjust the threshold down, we get more legitimate e-mail counted as spam.  If we adjust it up, we get more spam but the legitimate e-mail gets through.  The tricky part is where to set the threshold.  This is where I need your help.  Without your input I have no idea what is getting blocked or who is getting deluged.  For most of us, adjusting the threshold will only mean a few more spam per day.  For some, depending on the complexity of the spammers, it can mean hundreds more per day.  Keep in mind that just over spring break the system backed up by over a million e-mails.  Imagine if that amount of spam was traveling directly into folks mailboxes.  Having to sort through hundreds of e-mails to find the few that are legit, per day, can really dent your productivity.  Unfortunately, spam has developed from an annoyance to a serious problem.  Not only does it take up disk space but it can seriously hinder productivity.  All of us do e-mail triage on a regular basis.  When you have to sort through hundreds of spam just to identify your legitimate e-mail it takes time, energy and patience.  I really hate spam!  (I know, I am repeating myself).

 Let me give you an example from last week. 

In this time we have processed 1,261,372 messages, 567,383 (45%) of those were spam and 12,961 (1%) of those were suspected spam.  The suspected spam is based on our current settings.  I do my best to scan through the suspected spam to see how much legitimate mail might be getting held back but as you can see 12,961 messages per week takes a little sorting.  In other words, I can only do samples.  Thank you all for your help.  If you are really getting hit with spam (more then 15 a day) please let me know.  If you still have problems receiving mail I would also like to hear from you.  I can't answer every e-mail problem letter directly but the data does help.  Thank you all.  If you have any issues with mail that is supposed to be reaching you, please, please, please, include the addresses of the senders.  If it is too much spam then just send me a count.  If I need more info from you I will ask.  Thank you for your help.  We may not be able to slay the hydra but we will do our best to keep it at bay.