Skip to main content

New! A/B Split Test for SMS Broadcasts Explained

Learn how to test two versions of your SMS broadcast.


Not sure which version of your text will get more clicks? With A/B split testing, you can send two versions of an SMS broadcast to part of your audience, see which one gets the higher click rate, and send the winner to everyone else.


How A/B Testing Works

Step

What happens

1

You write two versions of your SMS: Message A and Message B.

2

Call Loop sends them to a test group, the percentage of your contacts you choose.

3

The test runs for the number of hours you set (1 to 24).

4

The message with more short link clicks wins and is sent to the remaining contacts.

Example: your list has 10,000 contacts and the test audience is 10%. 1,000 contacts take part in the test, and the other 9,000 receive the winning message.


Setting Up an A/B Test

  1. Go to Broadcasts and create a new SMS broadcast.

  2. In the message area, open A/B Split Test and turn the toggle on. A second message field appears.

  3. Write Message A and Message B. Both messages need a short link (see "Good to know" below).

  4. Under A/B Test Audience, enter the percentage of contacts to include in the test. The panel shows how many go to the test and how many wait for the winner (for example, "Test: 10%" and "Winner to remaining: 90%").

  5. Under Test duration, set how many hours the test runs before Call Loop picks a winner: 1 to 24 hours.

  6. Optional: turn on Audience split to change how the test group is divided between Message A and Message B. The default is 50% / 50%.

  7. Finish setting up your broadcast and click Create broadcast.


How the winner is chosen

Performance is measured only by short link clicks.

Result

What is sent to the remaining contacts

One message has more clicks

The message with more clicks

Equal clicks, or no clicks at all

Message A


See your results

Open the broadcast and go to the Overview tab. The A/B Test Result section shows, for Message A and Message B: total sent, clicks, click rate, unsubscribed and revenue. The winner is labeled Winner.

Good to know

  • A short link is required in both messages. You can't create an A/B broadcast until both messages include one.

  • Scheduling isn't available. If a broadcast was scheduled and you turn on A/B Split Test, advanced schedule and send later reset to the default and the broadcast is sent right away.

  • Change one thing at a time (for example the offer or the call to action) so you know what made the difference.

  • The winner is only based on clicks, not replies or revenue.

  • SMS and MMS.

Did this answer your question?