Lately, I’ve been fond of saying that a business is nothing more than an enormous collection of A/B tests. At a high level, we’re conducting a handful of big tests — Does market A or market B offer more opportunity for our firm? Do clients value service A more than service B? And, every day we’re conducting thousands of very small tests —Is work plan A more efficient than work plan B to complete this project? Is sentence A more compelling than sentence B to open this proposal?
We’re constantly testing these hypotheses both large and small. More often than not we do it unconsciously. But, we’re always doing it. And, the feedback these tests provide us help us to shape our vision, our strategy, and our daily decision-making.
For the past year or two, I’ve been talking about the importance of bringing a continuous improvement mentality to our clients’ digital marketing efforts. The idea is simple — the web is the most fluid, pliable medium in the history of world. Hence, we have the ability, or should I dare say the responsibility, to leverage its full potential — to regularly test different approaches, study the resulting data, refine and improve.
And, over the last 6 months, as a regular reader of this blog, you’ve unwittingly been part of a whole series of A/B tests. We’ve been testing email subject lines and prose, web headlines and call-to-actions, and website page structures. And, this post shares 3 lessons we’ve learned from 11 of those A/B tests, specifically as it relates to email marketing:
- Subject lines with numbers perform better than those without them.
- Longer subject lines (8-11 words) do better than shorter ones (3-7 words).
- Using slightly negative subjects and lead-ins seems to perform better than slightly positive ones.
So, let’s look at each along with some examples.
#1 – Subject Lines with Numbers Perform Better
This has been written about a lot in the past. So, this was no big surprise. People like lists. They’re easy to scan. And, hence, they’re well suited for the web. We tested number-driven subject lines against non-number subject lines 3 times in the last 6 months, and the subject line with the number won every single time.
#2 – Longer Subject Lines Perform Better
This one surprised me a bit. Given the rise of mobile devices, I think we all feel the pressure to be more concise with everything we do. For email copy, we’re concerned that people won’t read very long or very far. As for subject lines, we’re concerned about fitting into the reader window. But, what we’ve found is that longer, more descriptive subject lines (8-11 words) seem to perform better than shorter ones (3-7 words). We tested longer subject lines against shorter ones 4 times and the longer one won everytime. I think the reason is probably quite simple, a longer subject line has the ability to give us a better clue as to what the email actually contains. And, we simply don’t have time for ambiguity anymore.
#3 – Slightly Negative Subjects and Lead-Ins May Perform Better
If you read this blog regularly, you’ve probably noticed that I have a pre-disposition towards highlighting what’s wrong in the marketing efforts of most professional firms rather than focusing on what’s right. I can’t say this habit was grounded in any hard science. To be honest, it’s probably just a personality quirk. Like any consultant or advisor, I’m simply pre-disposed to see ways for our clients to improve. But, as it turns out, this may actually be a good strategy. We tested this 3 times by writing slightly negative subject lines and email lead-ins (content that’s more focused on what you could be doing wrong) and testing them against more positive ones (content that’s focus on what you could be doing right). And, 2x the negative messages won.
Applying These Findings
In the grand scheme of things, this is an infinitesimally small amount of A/B tests to draw any statistically significant conclusions. And, these outcomes are based on data derived from our marketing. So, they’re our numbers. Your audience may respond differently to your email marketing efforts. In the end, the goal wasn’t to get you to write a bunch of number-driven, 11 word subject lines with slightly negative body copy. The goal was simply to get you thinking about A/B testing, and how you might apply it to improve the performance of your web property. So, take a moment and think about the things you’re doing in marketing every day. And, think about how you might test them in a controlled A/B setting. Then, run some tests. The worst thing that can happen is you might bump up the click through rate a percentage or two on an email marketing campaign.
