Case Study: How Not Really Rocket Science Improved Email Open Rates 330% Using Smart Segmentation

Chris Bintliff, the mind behind Not Really Rocket Science, a forward-thinking digital marketing agency out of Madison, WI, conducted this rad strategy and is kind enough to share his findings with us in this guest post. Read on for some seriously good stuff!

A lot of businesses struggle with making the right marketing investments because it’s easy to forgo the important work of strategy and indulge in the strictly tactical, throwing adwords against the wall to see what sticks.

The tactical stuff is easier to understand; buy the ad, boost the post, download the list, send the campaign. Without broader perspective, tactics like these often just lead to wasted time and money, which frustrates everybody involved and can lead to more under-investment as leaders determine “Bah! this stuff doesn’t work.”

At my agency, Not Really Rocket Science, some of our favorite conversations are with people just like this—frustrated, confused, unimpressed, maybe even skeptical. This isn’t bad, though. It means there’s a lot of opportunity for growth!

Get the full case study

Build Better Relationships with Smarter, Segmented Marketing

Stark Financial (not their real name), a business that accommodates businesses needing to outsource some of their financial responsibilities, came to us frustrated with the poor performance of their email marketing. Like a lot of businesses, Stark wasn’t sure how engaged the people on their email list were, and their strategy didn’t include any sort of segmentation across their audiences.

They were relying on mass emailing a mix of engaged and unengaged people, hoping that if they sent enough, they’d manage to collect a couple clicks.

The results aren’t surprising. The open rates for their emails were in the low single digits and click-throughs were well below 1%. Yet, their email marketing was one of their most expensive marketing investments. I met one of their regional directors Pepper (also not her real name) at a workshop I delivered on email automation and Drip.

She was especially intrigued with my position that effective digital marketing, which can already feel uniquely sterile and detached, must seek to build trust and a relationship first. “We know how to do that as salespeople,” she said, “why aren’t we doing that with marketing?”

Working with Pepper to champion a fresh approach, we began a pilot campaign to engage a segment of Stark Financial’s audience with what we believe is a better approach to email marketing. We proposed a multi-step project, split into three sequential campaigns designed strictly to build trust.

The results of one campaign would inform the next, with our goal being to identify our most engaged audience, to whom we could better and more effectively nurture a sales relationship. The sales would come later, though, and our key performance indicators for this campaign were open rates and unsubscribes.

I want to share the approach and outcomes of our successful pilot so people like you and companies like yours can think differently about how you’re using Drip’s powerful features to build and deliver more impactful email marketing that’s results-driven and focused on relationships and relevance. Let’s go.

Start with a Clean Email List

Stark provided us with a list consisting of 1,056 cold prospects, or email addresses they’ve purchased, and only 206 warm subscribers, or people who’d actually engaged on their website and submitted their email addresses.

Job number one was to run both lists through Neverbounce to identify only the most valid addresses, eliminating unknown, catchall, or invalid email addresses. Even though Stark felt confident with its lists, particularly the 206 warm subscribers, the cold list came back as 65.7% valid while the warm list was slightly worse, at only 64.6% valid. While Stark delivered more than 1,200 email addresses to us, 825 were deemed valid and put through our first workflow.

Takeaway: Keep a clean list! Tools like Neverbounce are ideal for this, and with a direct integration with Drip, it’s easy to always start with clean email marketing campaigns. In addition, conduct regular pruning activities. Send emails to people who haven’t engaged in awhile and simply ask, “Hey, do you still want to hear from us?” and unsubscribe or remove those that don’t open or respond. It’s better to have a small list of highly engaged subscribers than a giant list of people who don’t care.

Start with the End: Creating a Results-Focused Approach

Next, we defined our strategic approach. I could write a separate article for each element of this approach, but at a glance, our project consisted of:

  • Three campaigns, consisting of brief, powerful educational emails. Each campaign consisted of multi-day workflows focused on distinct yet related topics addressing specific issues Stark’s clients often face.

  • Emails sent from Pepper and her personal email address. She provided us with the outline for each day’s topic, and we wrote the messaging to amplify her expertise while providing immediate value to the subscriber while keeping them personal.

  • No sales calls to action. We included a link to Pepper’s Calendly, and included her LinkedIn profile in her signature so any skeptics in the audience could check her bonafides. But remember, our goal wasn’t clicks, it was opens.

The business goals were a mix of concrete objectives and fact-finding. We wanted to:

  • Improve open rates (which started at under 6%) to be closer to the financial industry standard of 21%.

  • Identify the most engaged subscribers.

  • Identify the repeatable methods that seemed to work best and avoid those that might underperform.

Takeaway: Any new marketing initiative should start with knowing what you’re trying to achieve in a measurable way. This is the only way to make smart business decisions around your marketing. Ideally, these goals are connected to overall business focuses, like revenue growth or brand awareness.

Deliver the Message with Intent

Working with Pepper, we delivered insightful, expert advice through email marketing campaigns of five emails designed to be brief, powerful and compelling. The first campaign was focused on improving cash flow, the second campaign was about reducing costs, and the final campaign— reserved for the most engaged audience—involved techniques for improving profits.

Each message was laser focused on outcomes and practical how-to advice; we avoided the strictly conceptual or philosophical and, anticipating a largely cold or unknown audience, we skipped the usual “let me introduce myself” block of boredom that is a hallmark of poor cold outreach.

With subject lines like “Your financial reports are probably useless. Here's why.” we promised the subscriber a compelling and valuable lesson from each email. This helped people come to recognize Pepper’s emails as something positive and rewarding to watch out for in the inbox.

Takeaway: Practice engagement empathy with your audience by asking what is this like for the person on the other side of the screen? A common mistake in copywriting is working from ourselves forward instead of from our audience backwards - impress your audience right from the start with value and get to the point quickly.

Empower the Audience all the Way

It’s worth exploring in more detail exactly how we constructed the first email because it delivered significant value to both the audience and us.

As mentioned, we started by delivering a powerful lesson. No unnecessary warm-up, no wasted words. Even the subject line was a splash of cold water: “You’re Doing It Wrong: Saving Costs.” Then we jumped right in with this email:

Hi (first name):

Most companies trying to improve profit (“save money”) are doing it like we did a decade ago, before the Great Recession - by sharpening the pencil, cutting costs and slashing budgets. There are better ways, and what you don’t know is hurting your company’s financial health.

Then, we introduced two easy approaches to saving costs in the email’s instruction-minded approach. We ended this section with the question, “which one are you struggling with most?” with an invitation to reply or schedule a meeting to explore the issue, but the real purpose of the question was to create a point of reflection and introspection for the reader.

After the teaching, we told the reader in bold, “I have four more insights like this to share with you, and I’ll send you one each day [or every other day, depending on the workflow] this week. Here’s what’s coming up.”

Because we were sending message one on Monday, we were specific on the upcoming schedule, i.e., “Wednesday: How paying for things differently is like putting money in the bank.”

Then we encouraged the reader, “Don’t want to wait? Just click on an email above and I’ll send it to you immediately.” We used Drip’s trigger link capabilities and assigned one to each day of the week, allowing the reader to shortcut to the issue most relevant to them.

This approach might seem simple, but it delivers a powerful experience that builds trust:

1. We tell the reader to expect a message each day for the next four days.

2. We tell them exactly what to expect on exactly which day.

3. We empower them with a shortcut to go to their area of interest immediately.

On our end, the trigger link provides tremendous value. If someone opens this first email, “You’re doing it wrong - saving costs,” we can maybe make some safe assumptions that they’re interested or possibly struggling with saving costs. If they also click on a specific day/topic  / trigger link, we have an additional data point about something else that might be interesting or problematic for them.

This kind of insight can help guide all kinds of things, from making a sales conversation more meaningful to delivering more relevant emails or even helping to shape future strategies about what topics to build campaigns around.

We created Rules in Drip to further segment subscribers based on these behaviors, and we created additional Rules to recognize when somebody had clicked on Pepper’s LinkedIn profile signature link or Calendly link (even if they didn’t actually schedule a conversation.)

All of this insight helped us identify the most curious or engaged subscribers and alert Pepper to particular people she might personally follow up with.

Takeaway: Look for ways to empower your reader with questions or by inviting feedback and sharing information. In the case of an education campaign or email course, share the path ahead with your audience. Giving them options to go directly to a topic that’s most interesting will help them get more from the experience and help you get to know them better. Use Drip’s Trigger Links and Rules to progressively profile and segment subscribers based on their real-life behaviors.

Find the Right Timing: How Often Should You Send Emails?

In Campaign 1, focused on improving cash flow, we ran four parallel experiments. We took the cold list, consisting of 693 total people, and cut it in half. Group A received a daily email while Group B received an email every other day. We did the same with the warm list, splitting 132 total subscribers into daily and every other day.

The goal was to define our rules of delivery based on what worked best for our audience. We’d take these results to define if Campaigns 2 and 3 would run daily, every other day, or something else.

The results weren't what we expected:

Cold Email List

  • Open rate for emails sent every day: 20.6%
  • Open rate for emails sent every other day: 16.7%

Warm Email List

  • Open rate for emails sent every day: 24%
  • Open rate for emails sent every other day: 20%

Interestingly, the unsubscribe rate on the Warm list was larger on both the daily and every-other-day groups, presumably because these people were simply paying more attention to their inboxes.

An early finding after the first email campaign was that the open rate for our cold list daily was close to the 21% open rate goal we were hoping to achieve, while the warm list daily exceeded it. Already we were finding that a healthier list, meaningful content, and relationship-driven approach was resonating better with Stark’s subscribers.

Takeaway: Avoid making assumptions and let your audience deliver insights to guide your direction. Rather than just building and adhering to a pre-determined workflow schedule, we essentially surveyed our audience to tell us what worked best for them. Our subscribers clearly preferred daily touches, so we let that inform our process for the rest of our Project.

See the full case study

Build on What People Are Interested In

With Campaign 1 successfully showing us that we were on the right track with our messaging and approach, and that daily emails were the way to go, we waited a few weeks to start Campaign 2.

Our audience for this campaign applied the criteria that anybody who opened two or more emails from Campaign 1 would be be enlisted into Campaign 2, helping us whittle our list down to a more engaged audience. We customized Drip’s Lead Scoring values to do this, assigning different points values for opening emails.

Then, we generated the segment for our next campaign by defining “Lead Score Is Greater Than” our threshold score. This  resulted in 328 engaged people enrolled in Campaign 2, about 40% of our original list. The same approach was taken with our final phase of the Project - Campaign 3 was was sent to the 202 most highly engaged subscribers from the second campaign.

As our email list became smaller and more engaged, our email open rate performance reflected this interest with dramatic results.

Over the three campaigns, we saw average open rates increase from 20.3% in Campaign 1 to 44% in Campaign 3, while unsubscribe rates plummeted from 2.3% to .4%.

The best-performing email in Campaign 1 had an open rate of 28.6%, with Campaign 2's best performer having a 41% open rate, and Campaign 3's best boasting 47.3%. The worst performer in Campaign 1 was 14.2% while Campaign 3's worst performer was 39.1%. That means the improvement from the worst performing email to the best was a whopping 330%.

Takeaways: Build on success by repeating what works and refining with feedback and learnings. Introducing some scarcity by pausing between campaigns can leave your audience “wanting for more”—a great marketing position to be in for any company. Use Lead Scoring to continuously segment your list based on criteria of your choice. For us, this involved email opens, but it could be clicks, visits to a web page, or another factor relevant to your goals.

Finally, look for quality over quantity. In Stark’s case, Campaign 1’s 20.3% open rate meant that of the 825 people on its list, about 165 people were opening their emails, but maybe not with meaningful interest. Our final Campaign was sent to 202 people and opened by an average of 88 of them (44%). This is like having a party where half the people don’t really care to be there versus inviting only the people who can’t wait to spend time with you.

The Next Steps: From Conversations to Conversions

At Not Really Rocket Science, we like to say that 100% of the time, the thing you think is the issue isn’t actually the issue. In Stark Financial’s case, go way back to the beginning: Most of their email list consisted of non-subscribers harvested from purchased lists, a third of which were invalid anyway.

We can trace a lot of their email performance issues simply to not attracting the right people in the first place, a symptom of larger issues in their digital marketing. With enthusiasm high from the potential this project demonstrated, Stark is taking the smart next step as we work with them to improve their website design and performance, lead generation strategies, and how automation like Drip fits into a more comprehensive, holistic sense of their overall marketing and sales strategies.

Want to get started on your own segmentation strategy? Get started with a free 14-day trial of Drip.

Start my free Drip trial