A/B test lets you evaluate the success rate of your campaign to maximize the overall success rate to achieve efficient email marketing results.
Simply put A/B test works like this:
…but let's go step by step.
The options for A and B versions differ based on the account type you have. While Mailkit Base users can have different subject and sender but the content remains the same, users of Mailkit Syndicate and Agency can have also two different versions of the content as well as different templates. Also the campaign can have a different format and different image attachment settings.
Settings or A/B testing can be found on the campaign details page.
As soon as you click the Enable A/B test button you will get access to the A/B testing options.
Scheduling campaigns with A/B testing is similar to standard campaigns. You can schedule an immediate or unattended delivery at a specified time. In case the Schedule button is not available you have not set the senders for both versions of your campaign.
After the campaign with A/B testing has been scheduled you get an option to abort the final delivery - that means that while the evaluation group already received your campaign the remaining recipients will not.
Later on you can check the reports and analyze the campaign's performance as well as the performance of the individual versions within the evaluation group under the A/B Test tab.
If your campaign is time sensitive and you need to reach your recipients right away then A/B testing is undesired. If your mailing list is small (tens or low hundreds of emails) the A/B testing will have little to no impact.
There is no ideal evaluation group size. You must always consider the size of your mailing list and content of your campaign. Always calculate how many recipients will be participating. Example: If you have a mailing list with 1000 recipients and you choose to use 40% for evaluation the campaign will be sent to 400 recipients (200 will get version A and 200 will get version B) and 600 will get the more successful version after the evaluation.
Always consider the content and your campaign's goals. If it's more important to make sure your recipients open the email select the Opens as the method. If your campaign has a call for action and you want the version more visitors select the Visits.
The same goes for the evaluation period - there is a lot to consider. In general you need less time to evaluate the Opens and Visists, but the conversion rate on the other hand might take a few days to peak. The size of your mailing list needs to be taken into account as well - if your mailing list has 100k recipients an hour might not be enough to get the right feedback.