A/B Test

An A/B test allows you to create multiple versions of a campaign – for example with different subject lines, calls to action, or other content – and then automatically send those different versions to a subset of your total recipients.

The version that results in better metrics based on the criteria you define will then be sent to the remainder of your recipients after a designated period of time. Or if you'd rather manually select the winner, you can also stop the test and have it sent to the remaining recipients immediately.

NOTE: We recommend adding all of the recipients to your campaign before proceeding with the steps below so that you can easily see the sample size.

To create an A/B test:

  1. Go to the Content step of an in progress campaign.
  2. Next to the "Advanced" heading, just below the subject line, click the Show text link.
  3. Click the Create A/B Test text link.
  4. Click OK, and a second version of your campaign will automatically be created as an exact copy of the first.
  5. Modify the content of the second version as desired. Click the Add Version text link to add additional versions. You can also rename the default Version 1 and Version 2 by clicking the down arrow icon next to the version name.
  6. Click the A/B Test Settings text link.
  7. By default, the different versions of your campaign will be setup to be sent to equal segments of random recipients representing 20% of your total recipients. Drag the arrow icon to adjust the sample size. A minimum sample size totaling at least 1,000 recipients is recommended for optimum results.
  8. Select the criteria on which the winning version should be selected: unique view rate, unique click rate, or unique clicks on a specific link.
  9. By default, the A/B test will run for 4 hours after the campaign has been sent. You can enter a different duration if desired, and/or you can manually select a winner and stop the A/B test (see below).

To send tests of the different versions:

  1. Go to the Preview and Send step of the in progress campaign.
  2. Refer to the "Send a Test Campaign" section in the lower left.
  3. Click the Select Version down arrow and select the version you want to send.
  4. Click the Send Test Campaign button. If your campaign has more than 5 recipients, you will need to click the Select Recipient for Test Campaign button and then select up to 5 recipients.

To view the A/B test results:

After your campaign has been sent, you can view the results in real time via reports.

  1. Click the Reports tab in the main navigation menu.
  2. Click on the name of the campaign to view its details.
  3. In the upper right you will see "A/B Test" and the amount of time remaining as well as the "Select Winner" link.
  4. Click the A/B Test tab at the far right to view the metrics over time.
  5. At the end of the A/B test, the winning version will be sent to the remainder of your recipients.
  6. Or if you would prefer to suspend the test and immediately send a selected version to the remaining recipients, go to the campaign's reports and click the Select Winner link.