How to create a mutually exclusive group in Zoho PageSense | Help guide

Create a mutually exclusive group in PageSense

A/B testing is a great method to compare two or more versions of your web page to figure out which one performs better in front of your audience in terms of conversion. Sometimes, in order to speed up your process and maximize your marketing efforts, you might be tempted to run multiple A/B experiments at the same time. By default, when you are running several experiments in PageSense, it may be possible that a single visitor be exposed to more than one experiment simultaneously. This method of exposing visitors to multiple test pages works well if the pages are completely unrelated, or if you are tracking different goals.

However, in a few cases, as given below, exposing a single visitor to different experiments at the same time can actually interfere and skew your A/B test results:   

  • Running two or more experiments on the same page

  • Tracking the same goal on multiple experiments

  • Having visitors interacting between test pages on the same flow, such as in funnel analysis (for example buyer's journey funnel or newsletter sign up funnel).

To avoid this interference of data collected for one test by another, and help ensure that a single visitor who is part of one test does not become part of another one, you can simply create mutually exclusive groups in PageSense. This advanced setting in the tool helps you to group multiple tests together that are mutually exclusive and comes with the following benefits:

  • Your visitors are not exposed to multiple tests running on the same page

  • You can keep the reports clean by avoiding overlapping of visitors across different tests

  • Remove bias in test data and relate the change in conversion rate to the correct test.

IdeaImagine you run various tests during the Christmas season on your eCommerce website. A visitor comes to your home page and becomes part of test A. They then move on to the product page that shows them test B. Finally, when they add a product to the cart and try to checkout, they see extra coupon discounts on certain products wherein test C is in full effect.

Eventually, the visitor ends up buying something from your website and registers a conversion.

Based on the conversions obtained in each test, you will be able to figure out:

  • Which variation of the tests has highly contributed to the success of your Christmas campaign

  • If any of the variations in those tests interfere with the data and results of another

Perhaps the homepage test convinced the visitor to look at what offer you exactly provide, which later directed them to the relevant product pages.It may have been the product page test where the visitor saw new attractive arrivals and made them to add items to the cart. Perhaps it was the checkout page test that provided extra coupon discounts that had a greater impact compared to all other tests and made the visitor end up buying your item.

Unfortunately, you can't easily tell which A/B experiment got higher conversions and revenue on your site, as there can always be hidden, unrelated interactions between different test pages. The easiest way to handle this is by creating mutually exclusive groups.

 

Creating a mutually-exclusive group lets you to split your visitor traffic into the A/B test that you're running, and makes sure that each group of visitors participate in one test only. This way, there’s no possibility that a visitor can pollute the results of one experiment with the variant they. This is because, from a visitors' point of view, there’s only one experiment being runn and shown to them whenever they land on your page.

To create a mutually exclusive group in PageSense:

NotesNote: For more reliable results, we recommend that you start all the experiments in a mutually exclusive group after you've added them to the exclusion group.
1. Click the Experiment Settings button on the top bar, then click the New Group button under the Mutually Exclusive Groups tab.



2. Add a name and description to the exclusion group.

3. Click the dropdown under the Select experiment section, then choose the experiments that you want to add to your mutually exclusive group.
IdeaFor example, say you are planning to run different promotional campaign ideas during the Christmas season on your eCommerce site. In this case, by creating a mutually-exclusive group, you can add multiple experiments on your site to this exclusion group and ensure that no visitor will see more than one experiment that's in this exclusion group.


4. From the traffic dropdown, enter the percentage of the visitors you want to be allocated to each experiment.
IdeaBased on our example, you can choose your visitor traffic in such a way that you direct one experiment to 50% of visitors coming to your website, and another one to the remaining 50% of visitors. Learn how traffic allocation works in mutually exclusive groups.


5. Click the Plus icon to add another experiment to your mutually exclusive group.

6. Once done, click the Create Group button.
Similarly, you can create another group by clicking the New Group button.

To add new experiments in an exclusive group:
Select the group you want to add the experiment, click the plus icon then from the dropdown choose the new experiment and percentage of visitors to be included.



 

To remove finished experiments from the exclusive group:
Select the group from which you want to remove a finished experiment, then click the Close icon next to the experiment name.

Info

Info: If you delete an experiment from an exclusion group, visitors who were previously included in that experiment will be removed from the exclusion group. This means the visitors won't see any other experiments or campaigns in that exclusion group. However, if you have any experiment or campaign running outside this group, and if the deleted visitor meets the targeting condition, they could see this experiment or campaign.


To delete a mutually exclusive group:
Select the group you want to remove from the list,  click the More icon, then select Delete from the menu options, as shown in the figure.


To archive experiments in the exclusive group:

Select the group you want to archive from the list and select the Archive button as shown in the figure.


To find and unarchive experiments back to the exclusive group:

1. On the Mutually Exclusive Groups page, select Archived Groups from the dropdown to filter the list and show your archived exclusion groups.


2. Select the experiment you want to unarchive from the list and select the Unarchive button as shown in the figure.



How a visitor is bucketed to an experiment in a mutually exclusive group in PageSense

In PageSense, when experiments are not mutually exclusive, visitors are directly made part of every experiment that you are running on the website. For example, if a visitor sees Experiment A then they can also see Experiment B. This means the results for A and B can be skewed by the overlapping visitors. 


However, in the case of mutually exclusive experiments, PageSense chooses a random value to bucket visitors to one of the experiments in the group and then stores this value in the cookie. For example, if experiments A and B are mutually exclusive, PageSense chooses a random value to bucket visitors into either Experiment A or Experiment B. This method ensures that visitors won't see more than one experiment from each exclusion group.

How traffic allocation works in mutually exclusive groups in PageSense

Let's look at few examples to understand how allocating traffic percentage is done when experiments are mutually and non-mutually exclusive to each other in PageSense.
NotesNote: if you change the traffic allocation of an experiment in a mutually-exclusive group, returning visitors will continue to see the experiments and campaigns they saw before your change. New visitors will see experiments and campaigns according to your new traffic allocation.

Example 1: When two experiments are in mutually exclusive group
Imagine you are running experiments A and B in an exclusion group with the traffic allocations as follows:

  • Experiment A: 50% of traffic allocated

  • Experiment B: 50% of traffic allocated

In this case, PageSense randomly distributes about half of your visitors to Experiment A and half to Experiment B. This ensures that experiments don't overlap for the same users.


Now, if you add two more experiments to the exclusion group and reallocate traffic as follows:

  • Experiment A: 25% traffic allocation

  • Experiment B: 25% traffic allocation

  • Experiment C: 25% traffic allocation

  • Experiment D: 25% traffic allocation

Based on your updated traffic allocation, returning visitors will continue to see the same experiment they saw the first time: Experiment A or B. Returning visitors will not see Experiment C or D. On the other hand, your new visitors might see any of the four experiments. About 25% of your visitors will see each of Experiments A, B, C, or D.


Example 2:  When one experiment is overlapping and two experiments are in mutually exclusive group

Now imagine you're running three experiments in PageSense but one has overlapping visitors and two are in an exclusion group 'ABC' with the traffic allocation like below:

  • 100% of traffic falls in Experiment A

  • 50% of traffic falls in Experiment B in group ABC

  • 50% of traffic falls in Experiment C in group ABC.

In this case, all visitors will see Experiment A. Some visitors will see Experiment B, and some will see Experiment C. However, no visitors will see both Experiment B and Experiment C. This is because Experiments B and C are both in the same exclusion group, which makes them mutually exclusive. This eventually means all visitors will see Experiment A along with Experiment B or C.


We hope this documentation helps make the process easy for you. Please feel free to reach out to us anytime by dropping an email to support@zohopagesense.com if you need more explanation or have any questions.