How not to send a personalized email

After a satisfying day of coding a Goldmine-esque Automated Processes system to allow for user-defined outgoing emails on a recurring schedule, I setup the first timed email to go out the next morning–a simple, personalized “quick friendly note” to clients inviting feedback for our services. It was one of the more satisfying days I’d had in a while, because I completed a distinct module (in a relatively short amount of time, too).

My boss called me at 7am the next morning, and asked if we were spamming clients, in a no-nonsense tone dripping with controlled aggravation. Needless to say, this is not the most fun way to wake up in the morning.

I immediately logged into the server remotely and cut off all outgoing traffic on port 25, then checked the mail queue, which I found to have over a thousand messages in it. @#$! This is a very obvious problem if you know that we do not have over a thousand active clients. The number is much smaller than that, actually. Not only that, but the fact that the mail queue was actually that full indicated that there was a tremendous backlog, on top of what had already gone out.

The “quick friendly note” email was supposed to go out at 6am. At that point, it was about quarter after 7. The periodic “run” command was being executed every five minutes. I had just queued about 14 copies of the exact same email to over a hundred individual clients, many of whom had multiple recipients listed. Now that’s a good way to start your day!

Of course, the effect was worsened by the fact that this had to happen on an email that was very specifically crafted to look like it was personally written. Chock up one shattered illusion. I did at least get to the mail queue in time to clean out a lot of stuff before it had a chance to send. Thank goodness for computers all over the house, available at a moment’s notice. Literally.

The problem turned out to be that I had changed a MySQL “DATE” field into a “DATETIME” field to allow for finer resolution for tracking when the process ran last. That would have been fine if I had also changed the format of the variable it was comparing that value to–which had been carefully crafted as a DATE-like format. So, even if the dates happen to be the same, checking a DATETIME value against a DATE-formatted string would never, ever match. Hence, the process merrily sent out a new email every five minutes, blissfully unaware of the consequences. I made this change without re-testing the whole process, which is arguably my biggest mistake in this whole story.

To make matters worse, I found that of the five groups of clients we categorize into, one of the groups I had chosen (by pulling the numeric category ID out of the dusty recesses of my memory) was actually the FORMER CLIENTS category. Way to go, developer! I have never taken a memory improvement course. I’m not specifically great at it. I should know not to pull that kind of a stunt for convenience’s sake, ever. But, as always, hindsight is 20/20.

My boss is an amazingly pragmatic and understanding person, and although it was a dumb mistake on my part, he responded with a complete lack of anger and frustration. I did have to spend a few hours fixing the bug (quick) and sending out a new apology email (not so quick) to everyone who got multiple copies, and finally re-send the message to the clients who should have got it in the first place but didn’t.

If you value your career:

Always triple-check every test case for every step of the process after making last-minute adjustments, especially when dealing with PR-oriented systems. Never rely on memory only for old ID-to-value associations.