How to create and launch an A/B test in Zoho PageSense | Help Guide

Create and launch an A/B test in PageSense

A/B testing helps you analyze and observe how one version of a web page performs alongside another in front of your audience. On Zoho PageSense, you can quickly create, edit, and launch different versions of your web page, and test which one version performs better in terms of conversion rate. You can run an A/B test on the CTA buttons, images, head line messages, sub-headings, or any other web page element, to build the most effective website. 

Setup your A/B test

1. Click  A/B Test on the left pane, then click the  New A/B Test button. 

Enter a descriptive name for your A/B test and enter the URL of the web page you want to run the A/B test on.
For example, say you want to test the elements in the homepage of your website. So, we will name this A/B test 'Homepage A/B testing'. 

IMPORTANT: You can also run your A/B test across a group of web pages by enabling the Advanced button. This option also allows you to use different URL match types and include or exclude web pages from being tracked in a single A/B experiment. Learn how to create an A/B test using advanced URL targetingoptions.
3.  When you are done, click Create. 

Create a new variation using the visual editor

1. You'll see Variation 1 of your A/B test page in the EDITOR page by default.
PageSense provides you with options to create, rename, and duplicate variations in the Editor window. To create a new variation, click the Variation dropdown and then click Add New Variation.

2. Hover your mouse anywhere on the rendered web page to highlight the various elements of your page. 
The PageSense visual editor offers a range of options to edit the test, add styles, add or reorder images, adjust element positions, and hide content. The editor also provides options to edit or add custom CSS and Javascript code and switch between the 
Edit and Interact modes from the top bar to navigate between different pages on your website.

Once you are done making changes to your variation, click Save and then Next in the top-right corner to set up goals in your experiment.

Add a conversion goal

1. Click the + New Goal button in the GOALS page to add a new goal to your variation page.

You can choose any of the goal types from the list of options on the left pane:
  • Elements Clicks: Tracks conversions when a visitor clicks on a specific element on your web page. Learn more about creating an element click goal.

  • Link Clicks: Tracks conversions when a visitor clicks on a particular link on your web page. Learn more about creating a link click goal.

  • Page Views: Tracks conversions when a visitor visits a specific page URL. Learn more about creating a page views goal.

  • Revenue: Tracks conversions for the purchases made and revenue generated per visitor for each variation. Learn more about setting a revenue goal.

  • Time Spent on Page: Tracks conversions based on the average time spent by visitors on your web page. Learn more about creating a time spent goal.

  • Engagement: Tracks conversions when a visitor clicks on any of the elements or links on a web page. Learn more about creating an engagement goal.

  • Custom Event: Tracks conversions when a visitor achieves a specific event or set of events on your web page. Learn more about creating a custom event goal.

3. Click Save and then Next in the top-right corner to save the goals for your chosen variation. 

Select your audience parameters

1. Use the  Participation  slider in the AUDIENCE page to set the percentage of your audience who will participate in the experiment.
For example, you have designed a new home page for your blog site and you have a total of 10,000 people visiting your home page every month. Now, you want to A/B test your new home page on only 50% of the audience visiting your web page. In this case, you can reduce the participation percentage to 50% on the slider before starting your test to interpret your test results and understand the performance of your newly designed web page. You can target your experiment to a wider audience by increasing the participation percentage on your slider at any time.  

  • The included users are randomly assigned to either the original or one of the chosen variants in your experiment. 

  • By default, 100% of users who meet your targeting requirements will be included in the A/B test. It is generally better to start with a smaller audience when you are A/B testing for the first time. 

2. Choose a specific group of visitors you want to target this variation to based on the behavior, location, device and other parameters in the  Target Audience section. 
PageSense offers three categories: All Visitors (chosen by default), Preset Segments, and Custom Segments. If none of these audience types meet your requirements, click the +Custom Audience button.
For example, if you targeted an experiment to desktop visitors only, your original and variation pages will be shown to users who access the site from their desktop devices based on the traffic percentage set for each variation. Learn about the  different types of audience segments in Zoho PageSense.

3. Click Next to move on to the CONFIGURATION tab for the experiment.

Configure your testing options (experiment method, time duration, etc.)  

1. Use the Segment Traffic slider on the CONFIGURATION page to allocate the percentage of traffic that you want to enter into the variation and original versions of your A/B experiment.

For example, you decide to show the experiment to 80% of visitors (chosen in the Participation section of the AUDIENCE tab), then you segment those visitors so that 50% see the original site and 50% see the new variation. Now, of the total visitors who come to your website, 20% will not see the experiment at all and the remaining 80% will be split as a 50/50 ratio between the original and variation pages. Once the test period is over, the winner is determined based on the allocated traffic to each version.

Info: Traffic distribution expresses the probability that visitors will be shown a particular variation. If you have two variations and the original version and the traffic is split equally, each new visitor has a 33% chance of being shown each variation and the original. However, the variation's actual number of visitors shown on the REPORTS page probably won't exactly match the 33% probability, but over time, the number of visitors in each variation should approach this number. 
2.  Choose your preferred statistical method to run and evaluate your A/B test results in the  Set Experiment Method section. Zoho PageSense provides two types of experiment methods:
  1. Bayesian method: This method calculates the probability of A performing better than B. 

  2. Frequentist method: This statistical method only tells you if A will beat B, but not the probability of A performing better than B. 

    • Quick trends: This method is ideal for testing non-revenue related goals such as quick headline tests or CTA clicks. It provides quick results in a shorter testing time and gives a statistical significance of 90%. 

    • Optimal: This method is ideal for all kinds of experiments and provides fairly accurate results in a moderate testing time. It gives a statistical significance of 95%.

    • High accuracy: This is best for critical experiments that affect your revenue directly and provides highly accurate results with a longer testing time. It gives a statistical significance greater than 99%. 

3. Move on to the Experiment Activation Mode section to choose when you want to show the variation to your audience as follows:

  1. Activate when the page loads: This is the default option used to trigger the experiment as soon as the page loads. 

  2. Activate manually: This option is used to manually activate the test based on the custom code placed on your web page. Copy and paste the snippet code shown on the screen to the exact location on your webpage where you want to activate the experiment. 

    For example, you want to show your web page with new products and promotions when the user tries to close the page. Add your custom code inside the close function of your web page to trigger the variation page that asks customers to check out the extra discounts on offer. 

  3. Activate on satisfying a condition: This option is used to add a custom JavaScript code that returns a BooleanTrue or False value based on the visitor's behavior on your web page. When the Javascript code returns True, your experiment will be triggered automatically and show the experiment page. Note that the Boolean condition will be evaluated and monitored every 50 milliseconds on your web page. 

    For example, you want to test a variation page that shows a special newsletter pop-up to subscribers who scroll down half way through your blog post. 

Note: If you're using conditional activation for a multi-page experiment, make sure that the activation code is added to every page in the experiment. Otherwise, it will only activate on one page.

4. You can also enable or disable the following options in your experiment: 

  1. Generate heatmap reports: Enable the Heatmap button to see visitor engagement across different sections of your variation and original pages. Learn more about heatmaps here. 

  2. Schedule an experiment: Use the calendar to choose a specific date to launch your experiment.

  3. Connect with third-party tools: This option allows you to integrate your PageSense projects with other third-party applications like Google Analytics and Intercome to access and read your A/B test reports in your preferred tool. Learn more about integrating PageSense with third-party applications here.

5. Click Next to go to the SUMMARY tab.

Review and check for quality assurance

The quality assurance (QA) mode in the A/B test, is an easy and quick way to evaluate whether your A/B experiment pages look and work correctly before launching it to your audience. It functions like a preview tool that can be used to check if:

Your changes in the variation pages appear correctly:

  1. The right type of audience segments are included or excluded
  2. All the events and goals are tracking and converting correctly
  3. All user flows are captured from the page as expected
  4. The variation pages work perfectly in desktop, mobile, and tablet views, and across different browsers.
To use the QA mode in A/B test,
1.  Click the Launch QA Mode button in the SUMMARY page to open the experiment page with the QA window for the web page. 

2.  You can now evaluate the test using the following options:

  1. Select an experiment and variation in the A/B TESTS section to see if the changes appear on your variation page as expected.
  1. Click Define in the AUDIENCE SEGMENT section to check how your variations and experiences appear to different audiences and verify that events are firing correctly for the targeted audience segment.

  2. You can view and track all the interactions you make on your web page in the EVENTS section. These include goals achieved, elements clicked, and the audience targeted.

Info: Whenever you change the audience segment in QA mode, the Events section will be reset to the new audience segment conditions.
4. Click Generate Link (in the top-right corner) to share the preview link of your variations with your peers or internal stakeholders without launching your experiment. Note that this link will be valid for a period of 30 minutes only for QA testing.

Launch your A/B test

The SUMMARY page gives you an overview of your experiment's configuration options such as the number of variations created, goals added, targeted audience segments, and other settings you have made.

To launch your A/B test:
Click Launch in the top-right corner to launch your experiment.

Once your experiment is launched, a new REPORTS tab will appear next to the SUMMARY tab. 
The EDITOR, GOALS, VARIATION, and CONFIGURATION tabs will be hidden when the experiment is running. However, if you wish to make changes to your experiment, click the Pause button, make the required edits, and then click Relaunch to run the experiment again. 

Note: Ensure that the Zoho PageSense code snippet is installed on all the URLs of your A/B experiment. 

    Zoho CRM Training Programs

    Learn how to use the best tools for sales force automation and better customer engagement from Zoho's implementation specialists.

    Zoho CRM Training
      Redefine the way you work
      with Zoho Workplace

        Zoho DataPrep Personalized Demo

        If you'd like a personalized walk-through of our data preparation tool, please request a demo and we'll be happy to show you how to get the best out of Zoho DataPrep.

        Zoho CRM Training

          Create, share, and deliver

          beautiful slides from anywhere.

          Get Started Now

            Zoho Sign now offers specialized one-on-one training for both administrators and developers.

            BOOK A SESSION

                                    You are currently viewing the help pages of Qntrl’s earlier version. Click here to view our latest version—Qntrl 3.0's help articles.

                                        Manage your brands on social media

                                          Zoho Desk Resources

                                          • Desk Community Learning Series

                                          • Digest

                                          • Functions

                                          • Meetups

                                          • Kbase

                                          • Resources

                                          • Glossary

                                          • Desk Marketplace

                                          • MVP Corner

                                          • Word of the Day

                                            Zoho Marketing Automation

                                              Zoho Sheet Resources


                                                  Zoho Forms Resources

                                                    Secure your business
                                                    communication with Zoho Mail

                                                    Mail on the move with
                                                    Zoho Mail mobile application

                                                      Stay on top of your schedule
                                                      at all times

                                                      Carry your calendar with you
                                                      Anytime, anywhere

                                                            Zoho Sign Resources

                                                              Sign, Paperless!

                                                              Sign and send business documents on the go!

                                                              Get Started Now

                                                                      Zoho TeamInbox Resources

                                                                              Zoho DataPrep Resources

                                                                                Zoho DataPrep Demo

                                                                                Get a personalized demo or POC

                                                                                REGISTER NOW

                                                                                  Design. Discuss. Deliver.

                                                                                  Create visually engaging stories with Zoho Show.

                                                                                  Get Started Now

                                                                                                      • Related Articles

                                                                                                      • Create and launch a Split URL test in PageSense

                                                                                                        Use Split URL testing when you want to test multiple versions of your web page hosted on different URLs. This type of test is considered best when you notice several deficiencies in your existing page that might affect conversion. In such cases, it's ...
                                                                                                      • Set goals and launch your push notification

                                                                                                        The primary goal of any notification is to evoke a desired action from the users reaching your website. You can easily track this by creating different types of goals for your targeted audience on PageSense. These can trigger automated ...
                                                                                                      • Targeting specific audience segments in your experiment

                                                                                                        Audience segmentation in PageSense typically means dividing or grouping website visitors into subgroups and subcategories based on a common set of characteristics, behavior, needs, and values. For example, you can segment users based on the device ...
                                                                                                      • Create an A/B test using advanced URL targeting options

                                                                                                        Run better A/B tests and create better user experiences on your website using advanced URL targeting. This feature can be used to target and show your variations to a specific or group of pages on your site with different URL match types. Also, you ...
                                                                                                      • Different types of preset goals in A/B and Split URL test

                                                                                                        Setting up goals on your A/B or Split URL pages is the best method to track visitor behavior and performance of each variation during your experiment. PageSense allows you to create different types of preset goals on your variation page to track ...
                                                                                                        Wherever you are is as good as
                                                                                                        your workplace



                                                                                                          Watch comprehensive videos on features and other important topics that will help you master Zoho CRM.


                                                                                                          Download free eBooks and access a range of topics to get deeper insight on successfully using Zoho CRM.


                                                                                                          Sign up for our webinars and learn the Zoho CRM basics, from customization to sales force automation and more.

                                                                                                          CRM Tips

                                                                                                          Make the most of Zoho CRM with these useful tips.

                                                                                                            Zoho Show Resources