CRO hack #09: Beyond Inconclusive A/B Tests

CRO hack #09: Beyond Inconclusive A/B Tests



Despite having a hypothesis that checks all right boxes, there is still a high chance that your A/B test returns inconclusive results. However, this isn't a dead-end. In fact, there's still significant insight you can gather from this report. Here are a couple ideas to get you started:

Analyze micro conversions:
Say the primary goal of the test gets the highest conversions in a particular variation. If these numbers are statistically significant, then this variation is declared the winner, else the result is inconclusive. Now, instead of just looking into the primary goal's conversions, analyze the conversions of the secondary goals that you've set. You might find a significant jump in these micro conversions.

For example, visitors to the variation with the revamped content spend a lot more time on it. Or, a particular case study is shared more because of a new infographic. In both of these cases, your primary goal might have been a form submission. But these kinds of engagements can help your brand in the long run.

You can, however, use this new information to tweak the page further and run new A/B test on it—perhaps actively shaping the variation in a way that intuitively nudges the visitor towards the primary goal with minimal distractions.

Rework what defines a variation:
Just because one version of your hypothesis doesn't get substantial results doesn't mean you need to scrap the entire idea, just shift your perception of the hypothesis. 

Say you run a poll on your homepage and find out that visitors aren't signing up because they are unsure of your brand. So, to build brand trust, you create a variation of the page which has a section of trust icons. However, this test turns out to be inconclusive. 

Instead of discarding this hypothesis, think of a new approach to build your brand's trust: adding testimonials, links to case studies, or adding a section of certified ratings.


Slice and dice the test report:
It is crucial that you breakdown the behavior of specific audience segments after running a test. You might find that a particular segment if your audience is responding better to the changes you've made. 

For instance, many returning visitors convert if they are shown a product use-case video. Or, visitors from social media convert more if they land on the home page instead of the product page.

Next, you can personalize this change only for this audience type to maximize conversions.

In short:




Cheers, 
Tanya

          Zoho Marketing Automation

                  Zoho Pagesense Resources

                    Zoho SalesIQ Resources



                          Zoho TeamInbox Resources

                            Zoho DataPrep Resources



                              Zoho CRM Plus Resources

                                Zoho Books Resources


                                  Zoho Subscriptions Resources

                                    Zoho Desk Resources

                                      Zoho Projects Resources


                                        Zoho Sprints Resources


                                          Qntrl Resources


                                            Zoho Creator Resources


                                              Zoho WorkDrive Resources





                                                  Zoho Campaigns Resources

                                                    Zoho CRM Resources

                                                          Design. Discuss. Deliver.

                                                          Create visually engaging stories with Zoho Show.

                                                          Get Started Now