Repeat after me: testing is your friend, testing is your friend, testing is your friend. And it's time you start doing more of it! A/B testing your emails is how you get insights (big and small) about your customer base and what makes them open, click, add-to-cart, and checkout. If you are thinking, "yes, more of that," then you're in luck. Any marketer can master A/B testing, and we're here to help you learn the ins and outs.
So, what is email A/B testing?
Good question! A/B testing is a technical term for sending variations of the same email to different portions of your email subscriber list. Group A gets one email, and group B gets an email with one change. Then, looking at the engagement data (opens, clicks, conversions), you get info about what worked better and what your subscribers preferred.
When you use Drip, we'll help you A/B test a portion of your list and then automatically send the winning variation to the rest of your subscriber list. You can easily build A/B testing into your send strategy, so you are optimizing every email and learning about your customer preferences.
What's more, when testing offers, copy or imagery, the learnings you pick up in A/B email tests can often be applied to other marketing channels and lead to optimizations across your ecommerce marketing. That means engaged customers and more revenue for your business.
What metrics should I test?
When A/B testing, you are looking at Open Rate, Click Rate, and Conversion Rate. Each test should include only one variation intended to influence these metrics.
Open rate
Subject lines and preheader copy have the biggest influence on open rates, so they are the focus when looking to boost this metric. The Fall 2021 launch of Apple iOS15 somewhat skews this open rate data since it allows customers to opt out of sharing their open, location, and device info, and this trend will likely continue across other devices.
Which is to say, open rate is an important metric but best mediated by. Since Apple Mail reports anything received through their app as "opened," open rates may be somewhat inflated as adoption of iOS15 increases through the end of 2021. Keep this in mind when looking at data over the next few months.
Click rate
Click rate indicates that your email motivated a customer to want more. An A/B test to boost click rate will include a variety of elements such as:
- Images, especially those featured at the top of the email
- A line of copy or tagline
- Call to action (CTA) button copy
- Location of a CTA button
- Location of header copy or featured image
- Copy, including variations on headers, titles, and CTAs
- Different design templates that communicate the same message
Conversion rate
A/B tests that focus on Conversion Rate look at big picture influences. The factors that contribute to open rate and click rate also influence conversion rate since the customer journey begins the moment your email lands in your customer's inbox.
With that in mind, A/B test for conversion rate optimization by looking at variations that have the biggest influence on a customer's ultimate purchasing decision: overall offer, discount, or promotion; a creative campaign and the focus of its storytelling or positioning; or urgency tactics such as timebound offers or limited editions.
A/B test quick tips to get started
First things first, here are some key things you need to keep in mind with email A/B testing:
- Test only one variation at a time. If you are testing an image, don't change any other factors (such as location, size, copy, etc.). Multiple changes will muddy your metrics.
- Have a goal in mind and answer a specific question. When you start with an intention, your testing and results will be more clear. Don't just test for the sake of testing.
- Use the same types of customers, either a portion of your overall subscriber list or a portion within a customer segment. You don't want to test two different customer segments against each other.
- The bigger your test group, the more reliable the results. Aim for a minimum of 1,000-3,000 recipients for each test. Don't have that many to test? For now, focus on email best practices while building up your subscribers!
- Keep track of your test results, and let those learnings build on each other. This will help refine your email strategy and make your emails increasingly more effective over time.
What does an email A/B test in action look like?
The following is an example of a real A/B test with ecommerce brand Knickey that focuses on optimizing their abandoned cart workflow.
Workflows (and other automated emails) are some of the most important emails an ecommerce business will send and typically account for 10-15% of the email revenue that a business earns.
Abandoned cart emails are particularly important because 70% of people abandon their cart during the checkout process. That's a lot of revenue potential to pursue! Running an A/B test will help you determine what has the most promise in appealing to your customers (and potential customers) to help them make it all the way through the checkout process to an order confirmation page.
All three variations you'll see below are intended to help increase the click rate. When testing variations within the body of the email (it's 'content'), you are making changes to increase the number of readers who are motivated to click and land on your website.
Let’s consider three hypothetical variations:
Variation A includes product reviews, leveraging the power of social proof to help motivate your customer to complete their purchase. Variation B features defining brand values that the company thinks are a draw for their customer, specifically, the material used and how it is different and better. Finally, Variation C only shows the abandoned product, but neither of the other blocks gets to the point with a simple reminder would be best.
Ultimately, the Knickey found a winner and upgraded their entire abandoned cart automation to reflect that variation.
Unique results that fit your customers just right
Which variation is best? Something important to remember with A/B testing is that you are looking at how your unique subscriber list responds; the best result for your emails could turn out very different than another business’s subscriber list. That's why tests are so helpful at getting unique insights that reflect the preferences of your subscriber list.
Now that you know some A/B testing best practices, you’re ready to dive into the minds of your customers to learn what makes them tick…
Your turn to test with Drip A/B tests
With Drip single email split (A/B) tests, you can:
- Choose your winning criteria to test (open or click-through-rate)
- Test up to four subject line or email content variations in each campaign
- Set a test pool from your subscriber segment before you declare the winner that is sent to the rest of your intended recipients
- Select the time duration to run the test and then automatically send the winning variation to the rest of your segment
- See opens, clicks, and revenue for each variation on the Dashboard to optimize future sends
Ready to give it a shot? First, decide which metric you want to test: open rate, click, or conversion rate so you can determine if you'll be varying subject lines or elements of the email content (body). Then dive into your setup!
Should you A/B test every email?
Yes, and no. The short answer is that Drip makes it easy to build A/B testing into your email sending process, and as we said at the start, testing is your friend. When it’s this easy to incorporate testing into every send without hassle, there is no reason not to since it will help make your email marketing more effective (aka increase your earnings).
However, A/B testing also helps you learn what works best and then implement that strategy or approach going forward. It might always make sense to test a subject line or CTA, but some elements may only need to be tested on occasion.
Once you determine the branded elements like design or photo use that your customers click through the most, or what content and education they’re likely to engage with, then you may only test those factors on occasion.
A/B tests are an ecomm essential
By now, it’s clear why A/B testing is such an important part of ecommerce marketing. It should be equally clear that it’s a task that you are fully equipped to master like the true Ecomm Rebel you are.