Skip navigation

Advanced Workflows: How to A/B Test Email Funnels in Drip for Dramatically Higher Conversions

Screenshot of Drip's email campaign blueprints page

Note: The following guest post by Christoph Engelhardt is pretty technical, and was written by a crafty developer who’s pushing Drip’s Workflows to their limit. Don’t be intimidated; you can still get amazing value from Workflows without using a more advanced approach like the one outlined below. But this post shows you what’s possible with Drip.

Take it away, Christoph…

I recently wanted to conduct an experiment in Drip ECRM that required me to split test the content of an email. Except there is no built-in way to do that in Drip (yet).

In this guide, I will show you step-by-step how you can split-test email content.

Why Email Split-Testing is a Great Use of Your Time

If you’re building an email list, split-testing elements of your emails can provide a massive lift in your conversion rate. You’re capitalizing on compound interest.

Peep Laja from ConversionXL writes:

If you increase your conversion rate 5% each month, that’s going to be an 80% lift over 12 months. That’s compounding interest. That’s just how the math works. 80% is a lot.

So keep getting those small wins. It will all add up in the end.

Over time, continuously split-testing your emails will give you an unfair advantage over your competition.

And since leads on your email list have shown interest in your stuff (since they were the 1% to 3% of website visitors that actually subscribed), these leads are the closest in your funnel to converting into customers. Plus, when you find an email or sequence that works, it can drive growth for years. That’s why split-testing emails is a really good use of your time spent on marketing.

With that out of the way, let’s take a look at split tests in Drip.

How Drip’s Built-In Split Test Functionality Works

Drip comes with some split testing features built in. Here’s a picture of the UI:

You can find the built-in split testing under “Campaigns” in the menu bar. With Drip’s built-in A/B-testing tools you can test:

  • Subject line
  • “From” name
  • Delivery timing

If you plan to optimize your subject line or your “From” name in order to improve your email campaign open rates, you should absolutely use the built-in split testing features.

While this is great, it does not allow you to test and improve the core part of your broadcast emails: the content. But I’ll show you how you can split-test broadcast email content in Drip, anyway.

Note: Built-in A/B testing is currently only available for automated campaigns, or series of emails. 

Step #1: Create Two Broadcast Emails with the Content You Want to Test

To split-test email content, we need to send two broadcasts – one to each of the two “buckets” of people.

This way, we can write separate email copy AND Drip will tell us the open and click rates for both broadcasts.

To set this up, I will assume that you have people labeled with tags, bucket_a and bucket_b respectively. And don’t worry — I’ll show you how to tag all of your future and existing customers for split-testing with bucket_a and bucket_b randomly and with just a few clicks.

First off, to split-test email content, you need to create a new broadcast. Click “Broadcasts” and “New Broadcast Email” in your Drip dashboard.

I suggest you name your broadcast “Subject Line – Bucket A” so you can easily see which campaign was sent to which bucket. I also recommend to always use bucket_a as your control group. Leave bucket_b for your challenger.

In the following screen we select “Tags” from the dropdown list and fill bucket_a into the “Tag” field. We leave the second dropdown list at “include” and click the “Next” button in the lower right corner of the screen.

Next, write and schedule your email just like any other email.

When you have finished writing and scheduling the email to bucket_a, repeat the steps for bucket_b. Very soon, you’ll know how your emails compare in open rates, click rates, conversion percentage, and even revenue per email subscriber all with just some slight copy tweaks.

Pro Tip: Make sure that you send both broadcasts at the same time, so that differences in the delivery timing do not interfere with your experiment. You really have to isolate your experiment from undesired effects.

The resulting broadcasts should look like this (except that your broadcasts are “Scheduled”):

With this, we have figured out how to split-test email content with precision.

Step #2: Tag Your Recipients bucket_a and bucket_b Using This Secret to Sorting People in Drip

The most challenging task, by far, is sorting your leads and customers into buckets, so let’s tackle that one.

All you need to do is to tag people with either bucket_a or bucket_b randomly. How hard can that be? Turns out, we have to get a little creative.

In computer science, a common way to sort things randomly is to use some sort of timestamp. A timestamp is the elapsed time in seconds since midnight on January 1, 1970.

My first instinct in Drip was to somehow get the timestamp of when a new person is created in Drip’s database. Unfortunately that can’t be done as Drip does not expose that value to the users. I tried a few dead ends and eventually resigned to asking the Drip support.

After a bit of back-and-forth with the support team, I learned an important thing about Drip: You can use Liquid not only inside of emails, but you can also use Liquid to set custom fields in Workflows.

Below is a complete workflow that sorts people into two buckets. Notice the action “Set ab_bucket to {{now | timestamp | modulo: 2}}.” 

Modulo: 2 is how we sort users into random buckets.

It works like this:

When you set a person’s custom field ab_bucket to {{now | timestamp | modulo: 2}}, Drip uses the time: now | timestamp.

Modulo: 2 is a mathematical operation that takes an integer (the timestamp) and gives us the remainder, after dividing by 2.

In essence, it tells us whether the timestamp was an odd or even number.

This is the “coin flip” we use to bucket people into bukets A and B.

When you add this within a Workflow, you get a pretty random value for the custom field ab_bucket. After all, not everyone on your list is clicking on a link in the exact same second – are they?

Once the ab_bucket field is set, we can easily tag people with bucket_a if the custom field ab_bucket is set to 0. (Remember, this is the remainder after dividing the timestamp by 2. It will either be 0 or 1, depending on whether the random timestamp was an odd or even number.) We tag people with bucket_b if their custom field  ab_bucket is set to 1.

That’s it! You’ve solved the most important problem.

Now you can split-test emails like a mad scientist.

And if you want to sort people into A and B buckets quickly, Drip lets you do that, too.

Now, we need to perform a Bulk Operation on our existing list.

Note: Shortly after sharing a draft of this guide with the Drip team, they were kind enough to implement my feature request. Drip now allows users to access the time the person’s record was first created via the subscriber.created_at variable.

Step #3: Perform a bulk operation: Set a custom field for all subscribers to {{subscriber.created_at | timestamp | modulo:2}}.

Go into the “Subscribers” tab in your Drip dashboard. Select “Import/Bulk Ops” and click on “New Bulk Operation”.

Then we select “Choose a segment of existing subscribers” from the modal.

For our segment, choose all people who do not have the custom field ab_bucket set.

Then press the yellow "Next" button in the bottom-right of your screen. Next, select “Set a custom field” as the action. Set the identifier ab_bucket to the value {{subscriber.created_at | timestamp | modulo:2}}.

Schedule the task for immediate execution.

Et voilà! You now have everyone sorted into buckets for future split testing!

Step #4: Calculate Statistical Significance for Your Split Test

Okay. The last thing we need to talk about is the math you need to make an informed decision. Except that I won’t tell you the math instead I will just point you to the nifty little calculator: Split Test Calculator & Decision Tool

Just put in the number of people in each bucket in there along with the people who opened/clicked in each bucket. It will tell you conversion rates (which you already know from Drip) and also whether the results are statistically significant.

You could just go with the open and click rates that Drip show you, but you would not know whether you can be confident in the experiment to have worked out. So always take this extra step.

Conclusion

You’re all done! We’re set to conduct split tests in Drip. And I can finally start experimenting a bit — and finding the broadcasts and campaigns with the highest conversion rates.

So if you’ve ever wondered... 

  • “Should I add images in my emails?”
  • “What if we emailed twice as often?”
  • “Could I grow my sales just by tweaking our autoresponder subject lines?”

…now you can test it. With continuous improvements, and depending on your starting point, you can realistically double your lead-to-sale conversion rate over the next 12 months.

Christoph is the founder of LinksSpy.com and the author of the upcoming book, SaaS Email Marketing. To learn more advanced email marketing tactics, visit SaaSEmailMarketing.net.

A browser window showing the Drip Swag Shop

Want a more hands-on experience?

We love our blog, but how about a more hands-on experience? Take 2 minutes and see what your shoppers see when you use Drip.

Start my experience
x A browser window showing the Drip Swag Shop

Wanna see Drip from a fresh angle?

Craving a more hands-on experience? Take 2 minutes and see what your shoppers see when you use Drip with this interactive experience.

Start my experience