Getting Your E-Mail Delivered: Increasing E-Mail Deliverability

Summary: The author considers the six most common approaches to spam filitering and what (if anything) you can do about each of them. Then he outlines in detail word-trigger systems, specifically the new SpamAssassin rules that govern how e-mail is tested and determined to be or not to be spam. Words and phrases that flag spam in subject lines and message copy are listed. The author discusses collaborative network spam identification systems, as well as whitelists, blacklists, sender certification programs, SPF records, and e-mail delivery auditing services. Finally, he summarizes all this into 21 specific recommendations, which incorporate the newest SpamAssassin tests, all designed to help legitimate e-mailers increase their bulk e-mail deliverability.

This article is an excerpt from chapter 8 of the forthcoming Second Edition of The E-Mail Marketing Handbook. The chapter has been entirely re-written. It comprises 18 single-spaced pages and 7,000 words, describing in understandable (and only occasionally technical) language, how to increase the percentage of your bulk e-mails that get through spam filters to arrive in your recipients' mailboxes.

The most important e-mail marketing issue these days isn't what software or service to select. It is getting your bulk e-mail delivered in a climate of overwhelming spam. Some estimates are that spam, that is, unsolicited commercial e-mail (UCE) plus virus-generated bogus e-mails, make up 70% or more of the e-mails sent.

Internet Service Providers (ISPs) must scale their equipment infrastructure to handle all this unwanted e-mail. They try to filter out spam for two reasons: (1) spam retained in their system costs them money for extra storage space, and (2) customers are increasingly demanding ways to lessen the amount of spam. TV ads for various ISPs tout the anti-spam features of their systems. And if ISP filters aren't enough of a headache, corporations and individuals have their own filtering systems. But the filters often filter out "false positives," that is, opt-in e-mail requested by the subscriber.

If you're looking for ten easy steps to solve the problem, I'll disappoint you. Consistent 90% to 95% deliverability is very tough, even with the best of systems. But I will explain what is involved to achieve high deliverability, and some things you can do to improve your delivery rate.

Before I get into the details, I want to make sure you understand that I am not trying to tell spammers how to spam more effectively. I am directing these instructions to legitimate marketers who are have difficulty getting their opt-in e-mails delivered. You have a need to know how to get your message to your customers.

Types of Spam Filters

Let's look first at the six most common types of spam filtering systems. Note that most ISPs and filtering systems use a combination of these types, but its helpful to consider them one-by-one.

1. Challenge Responses Systems, such as SpamArrest (www.wilsonweb.com/afd/arrest.htm), require each unknown e-mail sender to answer a question or two, and then type in the letters or numbers contained in a non-machine-readable graphic. Then the message is transmitted to the recipient and you can be cleared to send messages thereafter. While effective for the recipient, these are a pain-in-the-neck for large newsletter publishers, but there's little to be done but to respond to the forms so your newsletter can go through. There's not much more to be said about these, except that publishers should check e-mail for these and respond.
2. Word Trigger Systems, such as SpamAssassin (spamassassin.apache.org) and McAfee Spam Killer (www.wilsonweb.com/afd/spamkiller.htm), scan an e-mail message and calculate a point score based on words and phrases that tend to be found in spam messages, such as "FREE" (with three exclamation points) and "Last Chance!" If the score is over a certain level, the e-mail is marked spam and is sent to e-mail prison. Note: Neither SpamAssassin nor McAfee SpamKiller are pure word trigger systems. (More on this below.)
3. Collaborative Network Reports, such as Cloudmark (www.wilsonweb.com/afd/cloudmark.htm), receive from members of a collaborative network reports on which e-mails they consider spam. When an e-mail receives enough spam votes, it is considered spam by the network and is automatically filtered into a spam filter. (More on this below.)
4. Whitelists and Sender Certification. Most ISPs have a whitelist of what they consider legitimate mailers whose e-mail they automatically let through to inboxes. With some ISPs, a mailer can apply to for consideration and review. In addition sender certification is possible through such services as Habeas and Bonded Sender which publish whitelists of their certified (and paid) members. (More on this below.)
5. Blacklists and Blocklists. Some ISPs check incoming e-mail sender domains or IP addresses against a list of known or suspected spammers, then rejecting senders appearing on the list. Some ISPs, on the other hand, use blacklists as only one among several factors they consider in determining whether a message is spam. (More on this below.)
6. SPF Records. Some ISPs and filtering systems check SPF records to determine whether the alleged sender is authorized to send e-mails through a particular domain. This is to catch the common practice of domain spoofing. (More on this below.)

Popularity of Spam Filters

To give you an idea of the usage of spam filters among small to medium businesses, in November 2003, 693 of my readers answered a brief questionnaire about how their spam filters were working. They comprised five groups:

(1) those with no spam filters who manually delete spam (18%), (2) those who rely on their Internet Service Provider's spam filters to catch spam before it gets to them (23%), (3) those whose company filters out spam before it gets to them (13%), (4) those who have installed a spam filter on their desktop computer to filter out the spam (22%), and (5) those who use filters on their e-mail program to filter spam by words and senders (24%). I asked participants to rate how their spam filtering system worked: excellent, pretty well, okay, and poor. Of the various approaches, participants were much happier with individual filtering than the other approaches. The graph below indicates which approaches were rated excellent by participants.

Here are the individual desktop spam filters were most popular.

I also asked readers to rate the effectiveness of their filter. I only graphed user ratings for spam filters which received more than three votes in the survey, so that the results would be more reliable. The rating scale of how the filter was working was Excellent -- 4, Pretty Well -- 3, Okay -- 2, and Poor 1.

Now let's consider several of types of spam filters in greater detail to learn how to avoid our messages being sent to e-mail purgatory.

Word Trigger Systems

According to my own informal study of anti-spam programs, SpamAssassin (http://spamassassin.apache.org) seems to be one of the most popular spam filtering system used by small businesses -- especially because it's free. It is usually installed at the mailserver level rather than on the recipient's desktop, thus it often filters e-mails for a dozen or a hundred employees in a small business. Another program that uses this type of filtering, at least in part, is McAfee SpamKiller (www.wilsonweb.com/afd/spamkiller.htm).

SpamAssassin has gotten a lot more sophisticated since 2002 when I reviewed ver. 2.4. In those days the program compiled spam scores based mainly on scanning for spammy keywords and phrases. The latest version, SpamAssassin 3.0, can (optionally) use six different approaches to determining whether a message is indeed spam (which may or may not all be active in any given installation).

1. Header tests that detect anomalies caused by falsifying the message's origin.
2. Body text phrase tests, scanning for keywords, keyphrases that are characteristic of spam.
3. Bayesian filtering (optional), a system of calculating the probability of a message being spam based on its contents -- and dynamically learning from spam and from good mail fed back to the system by users.
4. Automatic address whitelisting and blacklisting (default on, but optional), based on scores of previous e-mails from this sender.
5. Manual address whitelisting and blacklisting, where users can make their own list of good and bad e-mail addresses.
6. Collaborative spam identification databases (optional: DCC, Pyzor, Razor2), showing "fingerprints" of messages that have been identified as spam by members of these networks.
7. DNS blocklists, IP addresses of known or suspected spammers (i.e., blacklists).
8. Character sets and locales, such as excluding e-mails with Korean character sets sent to English-speaking recipients.

Separately, each of these methods can create false positives, but used together their accuracy is quite high.

As an e-mail sender, however, you can mainly influence only elements one and two, headers and body text phrases. After studying in detail the default rules set up in SpamAssassin 3.0, I've found a number of different strategies, delineated later in this chapter, that can help keep your spam score lower. Some of these are crucial; others are only of minor importance. But taken together they can help you get more of your legitimate opt-in e-mails through the obstacle course to your recipients.

Just a word. You shouldn't be paranoid about the rules. You can't rigorously avoid each possible spam trigger or you messages will be weird and artificial. You do have 5.0 points to work with before a message is considered spam at the default setting. These are just suggestions designed to keep your point count within allowable limits.

There are some changes from version 2.4 that I analyzed in 2002, which result in a few changed recommendations. In Version 2.4, newsletter subject lines with a date or issue number gave you some negative points to bring down your spam score. But for systems that have upgraded to 3.0.x, this no longer helps. Using an e-mail signature previously reduced your point score. No longer. "Quoted" reply text helped, too, but in 3.0.x it no longer does. A smaller message size used to help with some negative points. No more. Apparently, these characteristics are no longer reliable spam indicators so they are no longer part of the SpamAssassin algorithm.

SpamAssassin Subject Line Cautions

This chapter is focused on deliverability -- getting your legitimate e-mails past spam filters into your subscriber's inbox. In a later chapter, I'll discuss the importance of subject lines to getting your e-mail opened. These are two separate issues. But subject lines are often telling indicators of spam. Here are some things SpamAssassin looks for in a subject line. I've sorted these in descending order of points assigned (meaning the most serious offenses are shown first).

Too many raw illegal characters (such as Korean character sets) (2.9)
A gappy version of 'viagra' (2.7)
A gappy version of 'vicodin' (2.6)
A gappy version of 'xanax' (2.5)
Starts with dollar amount (2.4)
Lots of white space (2.2)
Indicates sexually-explicit content (2.2)
A gappy version of 'levitra' (2.1)
An English UCE tag (2.1)
A gappy version of 'cialis' (2.0)
A gappy version of 'soma' (2.0)
A gappy version of 'valium' (2.0)
"Your Bills" or similar (1.8)
Includes "life insurance" (1.8)
Guaranteed (1.7)
Contains "Your Family" (1.6)
Common spam sign (2 numbers) (1.5)
G.a.p.p.y-T.e.x.t (1.4)
Starts with "Hello" (1.4)
Talks about losing pounds (1.4)
Is indicative of a Nigerian spam (1.2)
"As seen" (1.0)
"Your own" (0.9)
A unique ID (0.9)
Is all capitals (0.8)
A gappy version of 'phentermine' (0.6)
"For only" (0.6)
Starts with Buy or Buying (0.6)
Has exclamation mark and question mark (0.2)
Starts with "Free" (0.1)

These rules affect spam points awarded for the subject line only. Now let's consider SpamAssassin rules for the message body.

Words and Phrases that Trigger SpamAssassin

This word list isn't complete by any means; it's more suggestive. But this gives you a good idea of what to be careful of. I've sorted this list in descending point order so you quickly see what words are "fined" the most.

Character set indicates a foreign language (3.2)
Tells you about a strong buy (2.9)
Attempts to disguise the word 'viagra' (2.8)
Online Pharmacy (2.7)
100% natural?! (2.6)
Deep discount medications (2.5)
Offers an alert about a stock (2.4)
Not registered investment advisor (2.2)
Talks about a bigger drive for sex (2.2)
University Diplomas (2.2)
What are you waiting for (2.2)
Claims to honor removal requests (2.1)
Money back guarantee (2.1)
Talks about a million North American dollars (2.1)
Claims you registered with a partner (2.0)
Confidentiality on all orders (1.9)
Has Yahoo Redirect URL or mailto address (1.2 to 1.9)
Mentions Generic Viagra (1.9)
Reverses Aging (1.9)
Talks about an E.D. drug using its chemical name (1.9)
Claims you have provided permission (1.8)
Describes body fat loss (1.8)
Stock Disclaimer Statement (1.8)
Possible porn - various types of feline (1.7)
Removes Wrinkles (1.7)
Talks about quotes with an exclamation! (1.7)
Compete for your business (1.6)
Free Preview (1.6)
Home refinancing (1.6)
Meet Singles (1.6)
No Claim Forms (1.6)
Score with babes! (1.6)
Talks about millions of dollars (1.6)
Attempts to disguise porn words (1.5)
Be your own boss (1.5)
Buy Direct (1.5)
Claims compliance with spam regulations (1.5)
Hardcore Porn (1.5)
Prestigious Non-Accredited Universities (1.5)
Resistance to this spam is futile (1.5)
Send real mail to be unsubscribed (1.5)
Talks about exercise with an exclamation! (1.5)
They have selected you for something (1.5)
Claims you can be removed from the list (1.4)
Claims you wanted this ad (1.4)
Contains 'earn (dollar) something per week' (1.4)
Contains mail-in order form (1.4)
Cures Baldness (1.4)
Freedom of a financial nature (1.4)
Get Paid (1.4)
No Medical Exams (1.4)
Adult Web Sites (1.4)
Amateur Porn (1.4)
SEC-mandated penny-stock warning (1.4)
Weird repeated double-quotation marks (1.4)
You can search for anyone (1.4)
Have you been turned down? (1.3)
Home refinancing (1.3)
Message has link to company offers (1.3)
People just leave money laying around (1.3)
Talks about free mobile phones (1.3)
Talks about 'starting now' with capitals (1.3)
Eliminate Bad Credit (1.2)
Why Pay More? (1.2)
Claims you can be removed from the list (1.1)
Contains 'Dear something)' (1.1)
Receive a special offer (1.1)
Generic Test for Unsolicited Bulk E-mail (1.0)
List removal information (1.0)
Possible mention of bill 1618 (anti-spam bill) (1.0)
Viagra and other drugs (1.0)
Amazing Stuff (0.9)
Consolidate debt, credit, or bills (0.9)
Information on mortgages (0.9)
Lowest Price (0.9)
Mail guarantees satisfaction (0.9)
Nasty, dirty, little, etc. (0.9)
Offers a full refund (0.9)
Save big money (0.9)
There is no obligation (0.9)
While you Sleep (0.9)
Hot, Nasty, Wild, Young (0.8)
Information on getting larger body parts (0.8)
Free Porn (0.8)
Talks about cellphone signal improvement (0.8)
Thousands or millions of pictures, movies, etc. (0.8)
Doing something with my income (0.7)
Lose Weight Spam (0.7)
Celebrity Porn (0.7)
Talks about Oprah with an exclamation! (0.7)
Best, Largest, Most Porn (0.6)
Fast Viagra Delivery (0.6)
One hundred percent guaranteed (0.6)
Dear Friend (0.5)
Describes weight loss (0.5)
Free Membership (0.5)
See for yourself (0.5)
Two or more drugs crammed together into one word (0.5)
Accepting credit cards (0.4)
As seen on national TV! (0.4)
Nasty Girls (0.4)
Requires Initial Investment (0.4)
Contains urgent matter (0.3)
Looks like mortgage pitch (0.3)
Mentions millions of (dollars) (0.3)
Something is emphatically guaranteed (0.3)
Talks about more with an exclamation! (0.3)
Talks about price per dose (0.3)
Avoiding bankruptcy (0.2)
Claims to be Legal (0.2)
Describes some sort of breakthrough (0.2)
Free express or no-obligation quote (0.2)
Information on growing body parts (0.2)
Nobody's perfect (0.2)
Talks about 'acting now' with capitals (0.2)
Talks about lots of money (0.2)
Talks about opting out (lowercase version) (0.2)
Asks you to click below in capital letters) (0.1)
Claims you opted-in or registered (0.1)
Click to be removed (0.1)
Contains 'free sample' with capitals (0.1)
Get a million e-mail addresses (0.1)
Guaranteed Stuff (0.1)
"if you do not wish to receive any more" (0.1)
Impotence cure (0.1)
Mentions an E.D. drug (0.1)
No such thing as a free lunch (0.1)
Offers a picked stock (0.1)
Offers Extra Cash (0.1)
Talks about opting out (capitalized version) (0.1)
Who really wins? (0.1)

This list isn't complete. It gives you topics, but not an exhaustive list of all the trigger keywords. A longer (but still incomplete) list is available in my analysis of SpamAssassin in 2002 (www.wilsonweb.com/wmt8/spamfilter_phrases.htm).

Collaborative Network Systems

As mentioned above, one of the types of spam filters is a collaborative network in which members of the network mark e-mails as spam. When enough members of the network do so, that e-mail is considered spam for all the members of the network, and automatically placed in spam folders or wherever the ISP or service sends such mail.

A full installation of SpamAssassin 3.0.x checks up to three such collaborative lists. E-mail messages that appear on such lists then receive substantial points towards the spam score.

• Vipul's Razor (http://razor.sf.net)
• DCC List (www.rhyolite.com/anti-spam/dcc/dcc-tree/dcc.html)
• Pyzor (http://pyzor.sourceforge.net/)

Many ISPs and filtering systems use one or more of these lists to help them filter e-mail. There's not much you can do if one of your bulk e-mails is marked as spam on one of these lists. Fortunately, there is something you can do with Cloudmark.

Cloudmark is perhaps the best known commercial anti-spam system that uses a collaborative network to determine which messages are spam. Cloudmark provides services to large and small businesses, as well as to individuals through Cloudmark Desktop (www.wilsonweb.com/afd/cloudmark.htm) which can be installed on Microsoft Outlook Express or Microsoft Office Outlook. I like and use Cloudmark and recommend it.

For marketers, newsletters, and e-mail service providers, Cloudmark provides the Cloudmark Rating for Newsletters, which verifies legitimate communications such as newsletters and mailing lists to ensure delivery. Once you have been approved, you place a Cloudmark e-mail address on your subscriber list, which then gives your messages greater consideration as being legitimate opt-in messages, even if some members block them as spam. For more information see http://rating.cloudmark.com

Whitelists and Sender Certification

Sender Certification

One of the most promising approaches to getting e-mail delivered seems to be some form of sender certification. That is, e-mail senders agree to certain legitimate e-mail policies and are monitored by the certification agencies. If they receive many unresolved complaints, their certification is revoked. At this point, sender certification has a two means of identifying qualified mailers.

1. Presence on a whitelist of all certified members, which is checked regularly by certain ISPs.
2. Identification in each e-mail header. This helps keyword-type spam filters, such as basic installations of SpamAssassin, to determine whether an e-mail should be sent through to the inbox without having to consult whitelists.

Thus far there are two emerging sender certification applications:

1. Habeas, Inc. (www.habeas.com) offers an annual license to mailers who certify that they use only confirmed opt-in procedures and agree to responsible e-mail practices. Costs range from $500 to $4000 annually (or more), depending upon what kind of e-mail sending role your business has. I currently use this, but it's difficult to quantify how effective it is. Habeas became famous for using a Haiku poem in a hidden header that was copyrighted and trademarked to discourage counterfeits. Thus if someone used the Habeas header they could be sued under both copyright and trademark laws. So long as the sender uses a dedicated IP address, Habeas also lists its members a whitelist that is used by some ISPs. In addition to SpamAssassin, Habeas whitelist is (or will be) used by USA.net, RoadRunner, NetZero, Juno, MSN, and MSN Hotmail, and others.
2. Bonded Sender (www.bondedsender.com). Originally developed by Ironport, the Bonded Sender program was recently acquired by Return Path. Mailers put up a bond to stand behind their pledge of legitimate e-mailing practices. Members then appear on a whitelist. Spam complaints against a company deplete their bond, which must be replenished. Costs include an application fee, a bond, and an annual license fee. Minimum one-time application fee is $375. Minimum bond for commercial organizations is $500 ($250 for non-profits). Non-profits pay no annual license fee. Annual fees for commercial organizations vary from $500 (for up to 500,000 e-mails per month) up to $10,000 for unlimited use. Organizations that use Bonded Sender must send e-mail from a dedicated IP address. Both Hotmail and RoadRunner are known to use Bonded Sender whitelists.

Both Habeas and Bonded Sender programs are beginning to gain some traction among IPSs and spam filtering systems. It remains to be seen if one or the other will emerge as the clear leader.

There's another sort of sender-based designation that is completely different, but deserves a mention. Hashcash (www.hashcash.org) provides proof of extra time invested. Since it requires significant computer time to "mint" a "token," spammers aren't likely to generate hashcash tokens, thus suggesting that users of the hashcash token tend to be responsible e-mailers. While this approach hasn't really caught on yet, hashcash tokens are currently recognized by SpamAssassin 3.0.x with a negative score, which tend to bring down an e-mail's spam score.

Whitelisting by Individual ISPs

Some ISPs are pretty hard to communicate with, but a few allow you to apply for whitelisting with them. Typically, the process requires an application in which you certify that you follow certain opt-in e-mail practices. If your application is accepted, the ISP will probably monitor how its members treat your e-mail messages. If you don't have a large percentage claiming that you send spam, you may be accepted onto their whitelist, meaning that you may send e-mail messages to their members without some arbitrary cut-off point. A big advantage is that your e-mail tends to be put in the subscriber's inbox rather than in a spam folder. ISPs of which I am aware that offer such a program include:

• America Online (AOL) Postmaster Info. Lots of good information available. Specific information on whitelist at http://postmaster.aol.com/whitelist/
• Yahoo! Mail. Contact http://add.yahoo.com/fast/help/us/mail/cgi_bulkmail requesting an application form.
• Verizon. Request form. http://www2.verizon.net/micro/whitelist/

Blacklists and Blocklists

Blacklists and blocklists are another approach to filtering e-mails. Blacklists are lists of domain names and IP addresses that have been reported to have been used by spammers. They are maintained by self-appointed spam vigilantes and referred to by desperate ISPs trying to filter out spam so it doesn't sap as many resources. Most ISPs don't rely on blacklists exclusively, but they can be an important factor. If your domain or IP address appears on a blacklist, it could significantly retard deliverability, especially if it is a widely-used blacklist.

It's easy to get on a blacklist. All it takes is an accusation. The blacklist administrators make no investigation. I am often accused of being a spammer though I have never spammed.

To get off blacklists, you need to contact each blacklist administrator -- if you can determine who they are and if they give you a way to contact them, which is often not the case. Sites where you can look up your status are Dr. Jørgen Mash's DNS database list checker (http://moensted.dk/spam/) or R. Scott Perry's Spam Database Lookup (www.dnsstuff.com/tools/ip4r.ch?ip=). Delivery monitoring services (discussed below) usually include some kind of blacklist monitoring as part of their service.

The difficulty, however, is finding out how to get off each blacklist. The best information I've seen on this so far is included in a service I subscribe to, Blacklist Monitor from MailWorkz (www.wilsonweb.com/afd/blacklist.htm), $67 annually, with a free trial account available. Blacklist Monitor e-mails me if my sites are newly listed on or newly removed from a blacklist. For each blacklist on which my site appears, the program includes notes on who to contact to request removal. Having said this, you may be on some blacklists because the owner (or former owner) of your domain's IP address block was reputedly friendly to spammers at some time in the past. This you can't do anything about except complain to your webhosting or mailhosting service to get their act together. Life isn't fair! 

We've looked at the ins and outs of the various methods used to identify and filter spam messages. Now let's look at what the poor e-mail sender can do. The first thing, of course, is to be aware of how poor your deliverability really is.

SPF Records

The final way that ISPs and spam filtering services sometimes detect spam is to compare the SPF record of the alleged sender to see that the sending service is authorized to send e-mail for this domain.

SPF (Sender Policy Framework) is an e-mail protocol that fights return-path address forgery and makes it easier for ISPs to identify spoofed addresses. For this reason, e-mail newsletter publishers who have published an SPF record stand a better chance of their e-mail being delivered. Is it a cure-all for spam? By no means, but it is an important step in the attack on spam. E-mail publishers can use SPF to get more of their e-mail delivered. To learn how to implement SPF, see my detailed article "SPF Helps Legitimate E-Mail Get through Spam Filters," Web Marketing Today Premium, November 2004 (www.wilsonweb.com/wmtp8/spf_howto.htm). It's pretty technical, but well worth doing if you do a lot of bulk e-mailing.

Tracking Inbox Delivery

It's one thing for your e-mails to be delivered to the ISP. It's quite another for them to get through the various filters and arrive (ideally) in your recipient's inbox or bulk e-mail folder.

E-mail sending programs "talk" to the ISP when delivering e-mail:

"Here's a letter for one of your subscribers," the e-mail sender will say.

"Wait a minute, let me check," the ISP will respond. Then a millisecond later: "Yes, we have that user on file. Send it on!"

At this point, so far as your e-mail sending program is concerned, the e-mail has been "sent successfully."

But has it been delivered to the actual recipient's inbox or bulk e-mail box? You may be getting some anecdotal feedback from subscribers. But the only way to determine the percentage inbox delivery particular mailing is to use a delivery auditing system.

Deliverability auditing programs all work essentially the same way. A mailing list is seeded with 100 to 150 e-mail addresses spread across a variety of domains. When each seed e-mail address from a particular mailing is received, the time is recorded. Then reports are compiled for each mailing showing percent deliverability by each ISP, delivery duration, etc. By tracking such reports, e-mailers can quickly spot which of the ISPs aren't delivering e-mails or stop delivery after receiving only part of a mailing. Based on the reports, mailers can take steps to improve deliverability.

If you are sending bulk e-mails through either a desktop mailer or CGI software on your server, you really need to monitor delivery. If you don't keep on top of this, you could miss delivery to thousands of opt-in subscribers and not even know it. Nearly all of the ASP hosted e-mail marketing services monitor delivery of their e-mails to the major ISPs, but you'll probably never see the results. They use it to spot delivery problems so they can fix them quickly. But if you're questioning the delivery record of your e-mailing service, you can employ your own delivery auditing service.

Here are the delivery auditing tools I am aware of:

E-mailAdvisor (www.pipersoftware.com) from Pathfinder Software (a Lyris company) also provides an excellent service. They were highly rated in JupiterResearch's Delivery Auditing Tools: Tactics to Improve E-mail Delivery (Concept Report, December 29, 2004). Pricing is not discussed on the website. ISPs monitored included adelphia.net, aol.com, att.net, bellsouth.net, cleari.com, comcast.net, cs.com, earthlink.net, excite.com, freenet.de, gmail.com, gmx.de, go.com, hotmail.com, iwon.com, juno.com, knology.net, libero.it, lycos.com, mac.com, mail.com, mailblocks.com, msn.com, netscape.net, netzero.com/.net, peoplepc.com, sbcglobal.net, socal.rr.com, spamcop.net, tiscali.co.uk/.fr/.it, t-online.de, usa.net, verizon.net, virgilio.it, virgin.net, web.de, and yahoo.co.uk/.com/.de/.fr/.it.

EnhanceRate (www.enhancerate.com). Provides testing of e-mails against various spam filters, including Brightmail and desktop filters. Pricing not discussed on website.

Pivotal Veracity e-Delivery Tracker (www.pivotalveracity.com). Provides auditing tools and comprehensive reporting on over 200 B2C and B2B ISPs and domains in 68 countries. Won ClickZ's Marketing Excellence Award 2005. Pricing not discussed on website.

Return Path Mailbox Monitor (www.returnpath.biz/delivery/monitor) monitors both delivery and blacklists. Return Path now sponsors the Bonded Sender program with Ironport systems. Pricing not discussed on website.

DeliveryMonitor (www.wilsonweb.com/afd/deliverymonitor.htm) from AWeber Systems is $37.95 per month. DeliveryMonitor's price is much lower than any comparable service I know of. It provides a 30-day risk-free trial (that is, a refund if you don't like it). Recommended for smaller businesses experiencing e-mail deliverability problems. I use it weekly to check newsletter delivery. It monitors aol.com, bellsouth.net, cs.com, earthlink.net, excite.com, hotmail.com, mac.com, mail.com, msn.com, netscape.net, netzero.com, sbcglobal.net, usa.net, verizon.net, worldnet.att.net, and yahoo.com.

Resolving Deliverability Problems through Relationships

I've been using a licensed bulk e-mail sending program hosted on my own server for nearly three years. It is certainly less expensive than ASP hosted services, considering the number of e-mails I send each month. However, I am seriously considering whether it is ultimately the best approach.

If low price is your main driver, by all means send your own e-mails using licensed software. But if deliverability is your main driver, your best bet is to engage the best ASP hosted e-mail marketing service that you can afford to send your e-mails for you. The reason? Relationships.

The better ASP services monitor deliverability constantly. If they find a problem delivering to a particular ISP, they'll get on the phone immediately to resolve the problem. They can do this for two reasons. (1) They have the phone numbers -- not an inconsiderable task these days. (2) They value their reputation and relationships they have developed with ISPs over the years. If they have a client who sends spammy e-mail, that client either changes quickly or is booted out the door. Not to do so would ruin the service's reputation for responsible e-mail sending -- and that is of utmost value.

There's no way I know to rate the quality of a service's reputation and relationships. But when you talk with sales people on the phone, ask about how they resolve delivery problems. Ask what care they take screening their own clients and uploaded e-mail lists. This will give you some clues to their concerns for deliverability.

21 Ways to Increase Deliverability

We've discussed the various methods used for spam filtering and various ways to increase your chances of getting your e-mails delivered. Now let me summarize this information into 21 recommendations. The first several are drawn from SpamAssassin's spam scoring system. Note: If you have trouble interpreting SpamAssassin's sometimes cryptic descriptions of what they allow and what they penalize for, please don't contact me -- consult their website (www.spamassassin.org/tests.html).

1. Use Capitalization Carefully

Capital letters are seen as "yelling" and spammy. Excess capital letters can cost you points. I had been using capitalized titles in text e-mail formatting until I found that I was being penalized for these. Since then, I've stopped using whole lines of capitalized type as headlines in my text newsletters. Instead, I limit capitalizing whole words as much as possible. SpamAssassin adds from 0.1 to 0.5 points when it sees "very strong 'shouting' markup."

2. Avoid HTML Messages that are Primarily Graphics

If your messages rely on graphics and few words to convey your message, they could gather a few points as suspected spam, so be careful. Relying heavily on graphics can also hurt you with recipients whose e-mail program preview window is set for a default of "graphics off," since they may not see your graphic message at all. Ideally, the graphic should accentuate the text message. SpamAssassin awards 1.1 to 1.6 points for e-mails with images that contain between 1200 and 2400 bytes of words. E-mail with images and only 400 to 1,200 bytes of words can get socked 2.4 to 2.9 points. HTML which has a low ratio of text to image area is penalized 0.1 to 1.7 points. In other words, you are penalized for e-mail messages that are mainly images. (We're not talking about attaching jpegs of your children to an e-mail message, but HTML with IMG tags.)

3. Beware of Trying to Trick the Spam Filter with HTML Obfuscation

SpamAssassin 3.0.x is smarter than its predecessors. Remember all the ways people have learned to write fr*e to avoid saying "free"? I doubt that it's worth the trouble any more. Some spammers use special HTML characters in the middle of a regular word to make it hard to match that word to a spam database. Others use a * or ^ or ' or . between letters to try to trick the filters. However, this practice, known as HTML obfuscation, can get you in trouble. SpamAssassin adds 0.4 to 1.0 points for up to 30% HTML obfuscation, and 2.3 to 2.5 points for 30% to 80% HTML obfuscation. It adds 1.1 points for "HTML tags used to obfuscate words." Sure, some e-mail get's through by using some kind of o.b.f.u.s.c.a.t.i.o.n, but filters are getting better at i-d-e-n-t-i-f-y-i-n-g and d*l*ting it.

4. Be Careful to Use Good, Standard HTML

Spammers often use faulty HTML to trick the spam filters. Thus bad or non-standard HTML tags have become a spam indicator that can cost you points. SpamAssassin adds 0.2 to 1.9 points for "bad HTML tags," from 0.2 to 1.0 points for "non-standard HTML elements." A missing HEAD tag can cost you 0.1 point. Random attributes in tags cost 0.4. Text after the </BODY> and </HTML> tags cost 0.3 points.

If you use a "marquee tag" (used to scroll text horizontally within a display box) you'll be assessed 2.2 points. E-mail that includes an HTML form to send e-mail messages costs you 1.8 points. A TBODY tag, a perfectly good HTML tag, costs you 1.0 point. All this may not seem fair, but this is what SpamAssassin these practices with spam and assesses points accordingly.

5. Watch Your Hyperlinks, URLs, and Domain Names

SpamAssassin gives links a good looking over, so be careful. URLs to .biz and .info domains that spammers sometimes use can cost you. Notice below that URLs with affiliate and tracking codes can hurt your spam score. An affiliate code could cost 2.2 points and a tracking ID number 1.8 points. You can get around the affiliate code problem by using Meta-Refresh redirect pages on your website. Here's the HTML code for such a redirect page which I use for an affiliate link to Dynamic Popup Generator. On my site the URL is: http://www.wilsonweb.com/afd/dpg.htm which doesn't look spammy at all.

<HTML>
<HEAD>
<TITLE>Dynamic Popup Generator</TITLE>
<meta http-equiv="Refresh" content="0;URL=http://bizzydays.com/x.cgi?a=r&id=1&aid=297&p=1&h=1">
</HEAD>
<BODY>
</BODY>
</HTML>

Note carefully the exact position of the quotation marks in this syntax, which I've highlighted in red so you can see them.

Since .biz and .info are often used by spammers, a URL with these domains can cost you 2.2 and 1.7 points respectively. URLs that contain words associated with porn can cost 0.9 to 1.9 points.

Since spammers often try to hide their true identify by using numeric IP addresses instead of domain names, these can be assessed 0.1 to 1.6 points. Links to likely spammer domains cost 1.5 points.

6. Use Large and Small Fonts Judiciously

I don't believe SpamAssassin's definition of large font sizes includes H1, H2, H3 heading tags. So use HTML heads and subheads rather than font tags to increase font size, since "large" font size costs 1.4 points and "huge" font size is 1.8 points. If copyright information in small type is hurting your point count, you might want to include it full size, since "tiny font size" costs 2.1 points. Using a font color similar to the background color to hide words can cost you 1.0 points.

7. Avoid Suspect Spam Phrases

This list is a long one, include earlier in this chapter. Don't talk the way slick marketers do. Use words and phrases that aren't as common, but still motivate the reader to action. Remember, obfuscating common marketing words can get you in trouble, too.

8. Be Careful with Subject Lines

SpamAssassin is particularly interested in subject lines, which I've discussed earlier. Avoid "like the plague" spammy subject lines that rack up SpamAssassin points. We'll talk in a later chapter about constructing subject lines that motivate your recipient to open your message. But for now, we're just trying to get delivered to your recipient's inbox.

9. Carefully Word Your Unsubscribe System

It seems ironic that legitimate opt-in e-mailers are penalized for having unsubscription information. But since so many spammers have bogus systems, it is apparently a spam indicator. Make sure your unsubscribe URL or mailto e-mail address doesn't include or refer to the word "remove." (0.1 to 1.0 points). If a URL contains the recipient's username and/or password you could be assessed 0.4 points. If you discuss "opting out" it could cost 0.1 to 0.2 points. You need to include ways to unsubscribe, of course, but avoid the phrase "click here to..." and substitute something like "use this link to ...."

10. Don't Mention Spam Law Compliance

It's unwise to claim that you observe all the spam laws. Only spammers say that -- and it can cost them 1.0 to 1.5 points.

11. Check Your List for Spam Flag Addresses

Occasionally, evil-minded people will add e-mail addresses to your list just to get you in trouble with the anti-spammers. Try scanning your database for e-mail addresses that start with abuse@, postmaster@, or nospam@. Sometimes an e-mail address will be inserted that subscribes you to an autoresponder each time you send out an e-mailing. You might scan for the word "subscribe" among your e-mail addresses (though this one won't affect you with the spam filters). Of course, if you have a confirmed opt-in system, you'll avoid this problem.

12. Monitor Challenge/Response Systems Requests among Returned E-mails

I am seeing a small but increasing number of recipients who use systems that block all e-mails except those that take the trouble to respond to an e-mail message and perhaps give a name and reason for the e-mail. Thus, it's important to monitor the mailbox for your "From" e-mail address to catch these.

13. Ask Subscribers to Put Your Address in their "Whitelist" or Address Book

As people subscribe, ask them specifically to place you in their address book (AOL), "safe list" (Hotmail), or "whitelist" (some spam filters). That way your e-mail will come directly to their inboxes. Asking may seem tedious or tiresome, but it may make the difference between your recipients seeing or not seeing your e-mail. I keep updating instructions to new subscribers on my site to reflect changes in the way ISPs do things, so check there for the most recent instructions (www.wilsonweb.com/arp/whitelist.htm).

14. Monitor Blacklists

ISPs and spam filter systems often check blacklists of known spammers to help them reject e-mails. If your bulk e-mail program's IP address or domain -- or yours -- gets on a spam blacklist because of complaints of spam, it will prevent some of your e-mails from getting to their recipients. Your e-mail service vendor should be actively working with ISPs and anti-spam services to keep an excellent reputation in the e-mail community and resolve any problems. But if your vendor fails to -- or caters to spammers -- your e-mail delivery can suffer. I use and recommend BlacklistMonitor from MailWorkz (www.wilsonweb.com/afd/blacklist.htm).

15. Use an ASP Hosted Service

Though I'm currently using licensed software to send out my bulk e-mails, it's getting harder and harder to get maximum deliverability with a limited staff to monitor deliverability and immediately get on the phone to solve deliverability problems. I now encourage marketers to use an ASP hosted service, sometimes called an E-mail Marketing Service (EMS), rather than do-it-yourself licensed software for bulk e-mailing.

16. Use a Delivery Auditing Service

Plan to use a delivery auditing service, such as DeliveryMonitor (www.wilsonweb.com/afd/deliverymonitor.htm), especially if you are using licensed e-mail software to send out bulk e-mail. It is vital to help you spot and correct deliverability problems. If you do use an e-mail service, you might check deliverability occasionally to make sure they are performing adequately for you.

17. Don't Send Bare HTML E-mails

It's sad to say, but some filtering systems consider bare HTML e-mails as a sign of spam. For my Web Marketing Today newsletter, I always ask the subscriber's preference for either HTML or text e-mails. But instead of sending bare HTML, I now send multi-part text/HTML MIME messages. These are delivered at a higher rate than bare HTML, similar to the delivery rate of text-only messages.

18. Use Confirmed Opt-in (Double Opt-in)

The time has come your company use confirmed opt-in for new subscribers. While the government may not require it (yet), then the free marketplace is clearly moving toward the higher standard of confirmed opt-in. When you're falsely accused of spamming, it's a whole lot easier to argue your case before an ISP or blacklist when you have a confirmed opt-in standard than if you don't. When you use confirmed opt-in, you'll also find it easier to secure an account with a respected e-mail service provider that values its reputation.

19. Purchase a Habeas or Bonded Sender License

If you do use a confirmed opt-in system and qualify for their standards, apply for a Habeas (www.habeas.com) or Bonded Sender (www.bondedsender.com) license.

20. Publish an SPF Record for Your Domain(s)

Since some ISPs and filtering systems check to see if you have a published SPF record, make sure your hosting service and your e-mail sending service have published SPF records correctly for your domains.

21. Use a Spam Checker to Test Your Message

We're now seeing some services you can use to test the spam quotient of your e-zines and e-mail offers before sending them out.

• SiteSell SpamCheck Report tests your message at no charge using tools such as SpamAssassin and sends you a report. Send your test e-mails to sales-spamcheck@sitesell.net  Be careful, however, to put the word "TEST" (in caps) as the first word in the subject. Otherwise, the system will delete the mail, thinking it is spam. Following the word TEST, add the subject line that would appear in the e-mail normally.
• Lyris ContentChecker checks your messages against a recent version of SpamAssassin. http://www.lyris.com/contentchecker/ With this tool you paste your content into a box in an HTML form.

I wish that I could guarantee that if you take all 21 steps, your legitimate opt-in e-mails will get through the spam filters. But I can't. I can't even get all my newsletters through, even though I've been e-mailing responsibly for a full decade. But these steps will certainly help raise your deliverability. Getting delivered is a constantly moving target, so don't become complacent. Keep monitoring your content, format, and "scores" so that your e-mails get to their recipients.