CRO hack #09: Beyond Inconclusive A/B Tests

Despite having a hypothesis that checks all right boxes, there is still a high chance that your A/B test returns inconclusive results. However, this isn't a dead-end. In fact, there's still significant insight you can gather from this report. Here are a couple ideas to get you started:

Analyze micro conversions:

Say the primary goal of the test gets the highest conversions in a particular variation. If these numbers are statistically significant, then this variation is declared the winner, else the result is inconclusive. Now, instead of just looking into the primary goal's conversions, analyze the conversions of the secondary goals that you've set. You might find a significant jump in these micro conversions.

For example, visitors to the variation with the revamped content spend a lot more time on it. Or, a particular case study is shared more because of a new infographic. In both of these cases, your primary goal might have been a form submission. But these kinds of engagements can help your brand in the long run.

You can, however, use this new information to tweak the page further and run new A/B test on it—perhaps actively shaping the variation in a way that intuitively nudges the visitor towards the primary goal with minimal distractions.

Rework what defines a variation:

Just because one version of your hypothesis doesn't get substantial results doesn't mean you need to scrap the entire idea, just shift your perception of the hypothesis.

Say you run a poll on your homepage and find out that visitors aren't signing up because they are unsure of your brand. So, to build brand trust, you create a variation of the page which has a section of trust icons. However, this test turns out to be inconclusive.

Instead of discarding this hypothesis, think of a new approach to build your brand's trust: adding testimonials, links to case studies, or adding a section of certified ratings.

Slice and dice the test report:

It is crucial that you breakdown the behavior of specific audience segments after running a test. You might find that a particular segment if your audience is responding better to the changes you've made.

For instance, many returning visitors convert if they are shown a product use-case video. Or, visitors from social media convert more if they land on the home page instead of the product page.

Next, you can personalize this change only for this audience type to maximize conversions.

In short:

Cheers,

Tanya

Zoho Desk Resources

Desk Community Learning Series
Digest
Functions
Meetups
Kbase
Resources
Glossary
Desk Marketplace
MVP Corner
Word of the Day

Topic Participants
Tanya Pandey

Sticky Posts
[Live Webinar] How to maximize your conversions using existing website traffic
Hello all, Did you know that on an average only 0.1% of your traffic converts? The rest are either heading towards your competitors or dropping off without being able to find what they came for. When you want to increase your conversions, it's natural
A/B Testing Idea #2: Make your headlines sound super cool.
Hello everyone, We're back with our second post in the CRO testing idea series. If you missed the first post, please check it out here: "Place irresistible and clear CTAs." In today's post, we'll look at how to effectively A/B test your headline content.
A/B Testing Idea #1: Place irresistible and clear CTA buttons
Are you wasting much of your time and effort A/B testing every single idea that pops into your head, yet you're not generating sufficient traffic to your website? Don't worry—we've got you. We'd like to introduce you to the A/B Testing Ideas learning
Did you know?#3: How to target and run your experiment based on custom audience segments in PageSense
Whatever kind of online business you're running, defining your target audience is the key factor to delivering relevant messages and further positioning your brand in the industry. Most marketing efforts revolve around this basic strategy of segmenting
Did you know?#2: How to filter your report by custom date range in PageSense
Do you want to quickly view your experiment analysis or responses over different time periods? Want to effortlessly compare the performance of your website by specific weeks, months, or years? Also, see whether your new marketing strategies or efforts