#DIGITAL | Why Measuring Subject Line Success by Opens Is All Wrong

Subject lines are incredibly important to the success of an email marketing program. Marketers understand that. It’s why email marketers A/B test subject lines more than any other email element, according to our State of Email Design report. However, what many marketers don’t understand is how to determine whether a subject line is successful.

Contrary to what you may have heard, opens are generally a poor measure of subject line success. That’s because the goal of most of your email campaigns is to drive metrics that are further down the funnel, like email conversions.

THE GOAL OF SUBJECT LINES

You’ve probably heard people say that the goal of subject lines is to get opens; that the goal of an email’s body copy is to get clicks; and that the goal a landing page is to get conversions. That sounds really reasonable, but it’s misleading.

This kind of thinking leads marketers to believe that a higher open rate is better because it means they’ve opened the top of their funnel up wider and more people have seen their email’s message. And if more people enter the top of their funnel, then they have more opportunities to convince those people to take action. So they write subject lines with the aim of attracting as many openers as possible—often using vague, overly familiar, or even misleading subject lines to accomplish that.

But that’s an overly simplified view of how audiences work. It assumes that all opens are equal—and they’re not. For every message you send, some members of your audience will be much more receptive to it than others. So you don’t want just anybody on your email list to open that email. You don’t want just anyone to enter the top of your funnel. You want your subscribers who are the most receptive to your call-to-action to open that email and enter the top of your funnel.

That’s the role of your subject line.

THE GOAL OF A SUBJECT LINE ISN’T TO GENERATE OPENS. IT’S TO GENERATE OPENERS WHO ARE LIKELY TO CONVERT. [TWEET THIS →]

TARGETING AND SELF-SEGMENTATION

In the age of digital marketing and Big Data, effective marketing is about reaching that receptive subset of an audience with a particular message. That’s why smart marketers take the time to determine which:

Facebook users to target with ads
TV shows to run their ads during
Conferences to sponsor
Homes to mail catalogs to
And on and on

Email marketers are fortunate in that our channel has lots of opportunities for targeting. Personalization and dynamic content allow us to insert tailored messaging into an email. And segmentation and triggered emails let us get particular messages in front of just the right people.

If you’re using any of those email marketing tactics—or doing targeting in other channels—then you already believe in the value of targeting, which is a key marketing tenet because it maximizes engagement and return on investment, while minimizing annoyance.

Most of the targeting that email marketers do involves matching content to a subscriber. The kind of targeting that a good subject line does is different in that it asks subscribers to match themselves to the content. In the same way that marketers give subscribers favorite topics to pick from in a preference center, subject lines give subscribers the opportunity to self-segment.

Of course, every email you send your subscribers asks them to determine if they are the right audience for this particular message. The only question is: How much effort are your subscribers going to expend to determine if the message is the right one for them?

A CLEAR AND DETAILED SUBJECT LINE IS A FORM OF TARGETING. [TWEET THIS →]

A good, subscriber-friendly subject line allows recipients to quickly self-segment and to qualify themselves as a lead for that email. This approach increases the chances they’ll read emails from you that are relevant to them, and decreases the chances they’ll experience “opener’s remorse” and feel like they wasted their time reading irrelevant emails from you.

THE COST OF OPENER’S REMORSE

It can be easy to get stuck in “an open at all costs” campaign-level thinking, but if you keep your focus on the subscriber-level relationship, then you know that not getting an open isn’t the worst thing that can happen to your email. It’s that your subscriber opens it and is so disappointed that they decide to end their email relationship with you.

Since receiving “too many emails” and “irrelevant emails” are top reasons that subscribers opt-out and report emails as spam, email marketers should always be weighing the potential for opportunity versus the potential for annoyance.

[Tweet this chart →]

IT’S NOT YOUR FAULT

If you were under the impression that subject line success should be measured by opens, it’s understandable. Everywhere you look there’s support for this point of view.

The internet is crawling with tens of thousands of blog posts about the top subject line styles, hacks, formulas, best practices, and strategies that increase, boost, or improve open rates. Many subject line A/B testing and subject line suggestion tools make their recommendations based entirely on open rates. And it’s easy to find copywriters and consultants who will say that the subject line is there to get people to open.

MEASURING SUBJECT LINES BY OPEN RATES IS THE BIGGEST, MOST WIDELY SUPPORTED EMAIL MARKETING MYTH. [TWEET THIS →]

So why the disconnect? There are a few reasons:

Desired Compartmentalization. Copywriters only want to be evaluated “on their part,” not the whole email experience, which they admittedly don’t control. However, marketers should insist that copywriters work toward a meaningful, shared brand goal. At the same time, copywriters should insist that marketers share detailed information on any audience targeting, email design and messaging, and landing page copy.
A Rush for Significance. When doing A/B testing, you reach statistical significance much more quickly and easily if you use opens instead of metrics that are further down the funnel like clicks or conversions. This allows vendors and consultant to advocate action more quickly. Unfortunately, in this case, quick results aren’t necessarily accurate results.
Poor Visibility. You or your vendors may not have good visibility into metrics that are further down the funnel. Find analytics tools that give you better visibility and then feed click and conversion data back into your A/B testing and other tools that help you with your subject lines.
A Lack of Enforcers. While Google’s search algorithm and Facebook’s new feed algorithm strongly penalize vague and click-bait headlines and titles, there are no penalties or email feedback loops that track back explicitly to subject lines. Those companies have recognized that their users dislike titles that aren’t descriptive, but too many email marketers still think open-bait subject lines are okay.

The myth of open rates as a success indicator is so pervasive that sometimes a less valuable outcome is declared the winner in A/B tests. That is what appears to have happened in this Auto Parts Warehouse case study, where a send time A/B test was done and the time that generated the highest open and click rates was declared the winner, even though it generated lower sales.

USE THE RIGHT METRIC FOR THE JOB

Open rates are a fine metric, but they’re a secondary metric at best. You’re much better off focusing on deep metrics—whether evaluating subject lines or anything else.

Consult our Email Metrics Matrix to better understand which metrics are best for judging campaign-, channel-, and subscriber-level email and business performance.