Create a Multivariate Test

Create a Multivariate Test

To create a multivariate test, follow the steps below. Test different combinations of page elements to find the variation that performs best.
Step 1: Create a New Multivariate Test
  1. Go to Optimize in Zoho PageSense.
  2. Select Multivariate Test.
  3. Click New Multivariate Test.


  4. Enter the following details:
    • Experiment name
    • Experiment URL
            
Specify the webpages where the experiment should run.
You can:
  • Include specific pages
  • Exclude certain pages from the experiment

    URL match types  

    PageSense provides different URL match types to help you target the right webpages.


    Match type
    What it does
    Example
    When to use
    Domain
    Targets webpages within a specific domain
    example.com
    When you want to target pages across a domain
    Simple URL Match
    Matches the URL based on the standard URL structure
    example.com/pricing
    For common URL targeting requirements
    Exact Match
    Matches only the specified URL
    example.com/pricing
    When you want to test one specific webpage
    URL Pattern Matches With
    Matches URLs that follow a specific pattern
    example.com/products/*
    When multiple pages follow a similar URL structure
    URL Regex Matches With
    Matches URLs using a regular expression
    example.com/products/.*
    For complex URL structures
    URL Contains
    Matches URLs containing a specific value
    /products/
    When the URL contains a common path or term
    URL Starts With
    Matches URLs beginning with a specified value
    example.com/products/
    When you want to target a section or group of pages
    URL Ends With
    Matches URLs ending with a specified value
    .html
    When pages share a common URL end
This ensures that the experiment runs only on the intended webpages. Preview to verify that your targeting conditions select the intended webpage. Review your URL targeting configuration and click create to apply the changes and move to the next step. After configuring your conditions, preview the URL to verify that the intended webpage is targeted.

Step 2: Create Variations in the Editor
After adding the experiment URL, open the Visual Editor to modify page elements.
To create variations:
  1. Click on an element on the page.
  2. Modify the element (text, color, image, etc.).
  3. PageSense automatically creates a variation for that element.
If you modify a new element, a new section will be created.


 
Add Additional Variations  
To create more variations for the same element:
  1. Locate the section for that element.
  2. Click the + icon.
  3. Modify the element again to create another variation.
The combination count updates automatically as new variations are added.


 
Rename Variations  
You can rename variations for better clarity. You can also rename the control version.
Examples:
  • Variation 1 – Menu color change
  • Variation 2 – Header text update
  • Variation 3 – CTA button color change
  • Control – Original green button
Device Mode  
You can switch between device modes when editing:
  • Desktop
  • Mobile
  • Tablet
This allows you to optimize page elements for different devices.


Step 3: Configure goals for a multivariate test
Goals help you measure the performance of your multivariate test. Choose a goal based on what you want to achieve and track how visitors interact with the different combinations of page elements.

Configure a goal

To configure a goal for your multivariate test:
  1. Click Add Goal.
  2. Select the goal type you want to track.
  3. Configure the required settings.
  4. Save your goal.


Once the test starts, PageSense tracks the selected goal to help you compare the performance of the different combinations.

Choose a goal

Select a goal based on what you want to measure:
  • Element clicks: Track clicks on a specific page element, such as a button.
  • Link clicks: Track clicks on links.
  • Pageviews: Track visits to a specific page.
  • Revenue: Track revenue generated through your experiment.
  • Time spent on page: Measure how long visitors spend on a page.
  • Engagements: Track visitor interactions with page elements.
  • Custom events: Track specific actions based on your requirements.
Choose a goal that matches your experiment objective to measure its performance effectively.

How are combinations measured?

PageSense tracks the selected goal for each combination of page elements and compares the results to help you identify which combination performs best. For example, if you test different headlines and button colors, you can compare how each combination affects your goal, such as button clicks or sign-ups.

Clarify what a combination meansExplain how results are compared

Step 4: Configure audience targeting
Audience targeting lets you choose which visitors can participate in your multivariate test. You can target specific visitor groups based on factors such as device type, location, traffic source, and visitor type.
To configure audience targeting for your multivariate test:
  1. Go to the Audience step in your experiment setup.
  2. Set the participation percentage.
  3. Choose a preset audience or create a custom audience.
  4. Configure the required targeting conditions.
  5. Save your audience settings.
  6. Click Next to proceed.

Why use audience targeting?

Audience targeting helps you test different combinations of page elements with the visitors most relevant to your experiment. For example, you can target mobile visitors to find which combination of headlines and buttons works best on mobile devices.

Participation

Participation determines what percentage of eligible visitors can enter your experiment.
By default, participation is set to 100%. You can reduce this percentage to limit the number of visitors included in the test.
Example: If participation is set to 50%, half of the visitors who match your targeting conditions can participate in the experiment.

Preset audiences

Preset audiences let you target common visitor groups without creating conditions manually. You can choose from options such as:


  • Direct visitors
  • Paid campaign traffic
  • Organic search traffic
  • Social traffic
  • Referral traffic
  • Mobile and tablet visitors
  • Desktop visitors
  • New visitors
  • Returning visitors

Create a custom audience

If the preset audiences don't meet your requirements, create a custom audience using your own targeting conditions.
To create a custom audience:


  1. Enter an audience name.
  2. Add a description, if required.
  3. Select a targeting condition.
  4. Configure the condition.
  5. Add more conditions if needed.
  6. Save the audience.
You can target visitors based on conditions such as URL, traffic source, location, device type, browser, operating system, and visitor type.
Example: Target mobile visitors from India to test which combination of page elements performs best for this audience.

Step 5: Configure experiment settings
Experiment settings let you control how visitors are assigned to different combinations, how results are measured, and when your test runs.

Configure experiment settings

To configure your multivariate test:
  1. Configure traffic distribution.
  2. Select an experiment evaluation method.
  3. Configure the activation mode.
  4. Enable heatmaps if required.
  5. Set the experiment start and end dates, if needed.
  6. Click Next to proceed.

Segment traffic

Traffic distribution determines how visitors are assigned to the combinations in your multivariate test.


Manual distribution

Use manual distribution to control how traffic is divided between the combinations.
Example: If your test has four combinations, you can distribute traffic equally so that each combination receives 25% of the allocated traffic.

Auto distribution

Use auto distribution to let PageSense adjust traffic allocation based on performance, if this option is supported for your multivariate test.

Set experiment method

The experiment method determines how PageSense evaluates your test results.


  • Bayesian method: Estimates the probability that one combination performs better than another.
  • Frequentist method: Uses statistical significance to evaluate whether the observed differences are meaningful.
Choose the method that best suits your testing requirements.

Statistical significance level

If you select the Frequentist method, choose a statistical significance level based on how confidently you want to evaluate the results.


  • Quick Trends: Helps you identify early performance patterns.
  • Optimal: Balances testing speed and accuracy.
  • High Accuracy: Requires stronger statistical evidence and may need more time and traffic.

Experiment activation mode

Choose when your multivariate test should start showing the combinations to visitors.


  • Activate when the page loads: Displays the assigned combination when the page loads.
  • Activate manually: Starts the experiment through a custom implementation.
  • Activate on satisfying a condition: Starts the experiment when the configured condition is met.

Enable heatmap

Enable heatmaps to understand how visitors interact with the different combinations in your test. Heatmaps help you identify click patterns, scrolling behavior, and areas that receive more or less attention.

Schedule your experiment

Schedule your multivariate test to control when it starts and ends.

  • Launch this experiment later: Schedule the test to start at a future date and time.
  • Start date: Set when the experiment should begin.
  • End date: Set when the experiment should stop.


Best Practices  

  • Run multivariate tests on high-traffic pages.
  • Limit the number of variations to avoid excessive combinations.
  • Focus on elements that directly impact conversions.
  • Allow experiments to run long enough to collect sufficient data.



We’ve designed this documentation to guide you every step of the way. If you need further assistance or have any questions, don’t hesitate to contact us at support@zohopagesense.com - we’re always here to help!