Split testing helps you identify the most effective variant of your email campaign by comparing different elements such as subject lines, from names, creative content, and from addresses. Dotdigital offers three split test types to suit different goals: you can test on a sample and automatically send the winning variant to the rest of your audience, split your entire audience evenly across all variants, or set a control variant to benchmark against. Each method provides reporting to help you understand performance and improve future campaigns.
Before you start
Things you need to know:
Make sure you are familiar with the basics of creating and sending email campaigns before using split testing.
Create a split test
Configure the split test settings during the Create step of the campaign building process.
To create a split test email campaign:
Go to Campaigns > Email and either select NEW CAMPAIGN to create a new email campaign, or select an existing campaign to edit.
If you choose to edit an existing campaign, it may open on the Content step of the campaign creation process. Select the Create step from the breadcrumb menu in the top right to configure your split test settings.Select SPLIT TEST SETTINGS.
In the Split test settings side panel, enable the split test features you want to use.
Subject lines - critical for contact engagement.
Friendly name - find out which from name your contacts respond to most.
Creative - identify the most effective design or content for your campaign.
From address - see which originating email address your contacts respond to most.
The From address option only appears if you have more than one from address configured on your account.
Select APPLY to save your settings and create a split test email.
Use split test emails
With split testing enabled, two different variants of your campaign are available by default, accessible by tabs in the top left.
Select + to add more variants, up to ten in total.
Select the X symbol to delete a variant.
For split tests based on subject line, friendly name or from address, you see the variant tabs on the Create step. For creative, you see the variant tabs on the Content step.
You can only add variants on the Create step, not on the Content step.
Select a tab to see that campaign variant. Enter your content for each variant, then select SAVE or SAVE & CONTINUE.
Configure and send a split test campaign
After you create and save your split test variants, continue to the CONTACTS step. Set up who receives your campaign, when it sends, and how your split test runs.
1. Choose your audience
Select which lists, segments, or marketing preferences you want to send your campaign to. At this stage, you can add or exclude recipients to tailor your audience.
2. Choose send time
Decide when your campaign is sent:
Immediately - your campaign sends as soon as setup is complete.
Scheduled - set a specific date and time for your campaign to send.
Using send-time optimization - this predictive feature powered by WinstonAI sends emails at the time most likely to be opened by each contact. This option isn’t available for split test campaigns.
3. Set up split test
Once you’ve chosen your audience and send time, configure how your split test runs. There are three configuration options:
TEST THEN SEND WINNER
Send each campaign variant to a sample of your audience. The system measures performance and automatically sends the winning variant to the rest of your audience.Select the percentage of audience to test - use the slider to choose what portion of your audience receives the test variants - up to a maximum of 60%.
Set test duration - choose how long to measure responses before picking a winner, in hours.
Measure success by - select how the winning variant is determined, by Opens, Clicks, or ROI.
If you don’t see the ROI option, it may not be available in your account. This depends on your enabled features and package.
Test summary displays variant count and test group sizes as you adjust settings.
SPLIT AUDIENCE EQUALLY
Divide your audience evenly among all variants.Your campaign audience is evenly divided across all variants, with each contact receiving one version. There’s no automated winner send, you can review all campaign results to identify which performed best.
Example: With 4 variants and 8,000 contacts, each variant is sent to 2,000 recipients.Test summary shows how your audience is split.
Use reporting to compare clicks, opens, etc. after the campaign.
SET CONTROL VARIANT
Designate one variant as your control, typically your standard variant. Once your control is set, the remaining variants are split equally over the rest of the audience. This ensures a fair comparison between your control and test variants.
No automatic winner is sent, you must analyse the performance after sending to decide which is most effective.
The control variant provides a baseline for comparison against the test variants.Select control variant - choose which variant is the control.
Control audience percentage - set the proportion of your audience to receive the control variant.
Test summary updates to show variant and control audience allocation.
4. Set up ReMail
To learn more, see Use ReMail to resend a campaign to non-engaging subscribers.
Save and send your split test
When you’re happy with your options, select SAVE & CONTINUE.
You see the campaign settings displayed for review. Each set of variables are shown as a different summary tab.
Once you’re ready to send, or schedule your send, select either SEND CAMPAIGN IMMEDIATELY or CONFIRM SCHEDULED CAMPAIGN depending on the option you chose for the campaign send time.
A confirmation message is displayed. Select CONFIRM.
The campaign is moved to the Outbox for processing, and this is shown by the indicator displayed under the Status column. Once the send is completed, the campaign moves to the Sent tab.
Split test report
To learn more about campaign reports, see Email reports.