A/B/X Tests of On-site Actions

A/B/X tests of On-Site Actions is a solution that allows you to select the best performing variant of your action.

Actions on sites feature allows you to display dynamic website content adjusted to users’ traits and behaviours. Hence, the proper message will reach a clearly defined target group of visitors in real-time. The A/B/X testing feature enable marketers to select the most successful version of their message.

This feature allows you to create up to 5 different versions of the same message and display it to 5 different test groups of users.

On-site Actions test - How to get started

In order to activate the test, move the slider to the Yes position in the Settings of your campaign.

 As you turn it on, the Action step will change its name to Test.

When the criteria of the campaign are met for the first time by a contact, the system will randomly pick one of the variants. The next contacts may receive other variants of the action according to the percentage value you defined.

Version key

You can test up to five versions of an action. If you want to create a new version, click the Add new version button. In every version the following elements have to be specified:

Version key it will be visible in reports and activity parameters related to pop ups and Javascript code. It may be regarded as the name of the version, so it is worthwhile to set it up in such a way that will allow you to find the proper version at any time.

Action selection

Action - select the action that is to be performed after the criteria are met. Such actions have to be prepared in advance, so if you want to compare three versions of a pop up, you have to prepare it earlier. You may choose one of the following:

• Popup - the selected Popup creative will be displayed on the page
• JavaScript - JavaScript code that will be executed once the conditions are met
• Activity - The selected activity will be added to the contact
• Send WebPush - The selected message will be sent. Only communications triggered by API can be selected from the list
• Send MobilePush- The selected message will be sent. Only communications triggered by API may be selected from the list

Frequency of version selection

Frequency - define how often this version of the Action is to be performed. The sum of all versions has to be equal to 100%. Use the slider to set the value for which % of contacts meeting the criteria the system is to draw a given action.

For example, for tests of 2 different actions, you can set the values as 20% and 80% respectively, the sum being 100%. After launching the test, the first person will receive the first version while the next 4 users will receive the second version.

After performing the test, the results will be visible in the report generated by the system. In the case of e.g. a pop-up, the report will contain data on the number of impressions, clicks and closings of the message. After comparing the results achieved by different versions, you may run a campaign that will employ the variant of the message which was the most popular among recipients.

Reports from A/B/X tests in Actions on the Sites

Reports from A/B/X tests can be generated for active and completed actions clicking the [report] button next to the appropriate A/B/ X tests on the Actions list on the page.

 In the report, you can choose the previously defined [test version key] to check statistics, e.g. for the selected pop up, and the aggregation method (daily, monthly, hourly) or in a given period of time.