CRO hack #12: QA checks you should conduct before launching your A/B test

CRO hack #12: QA checks you should conduct before launching your A/B test



Prevent spotty user experience by running a thorough quality assurance(QA) check before launching your A/B test. Evaluate the test's behavior under variation-specific edge-cases. Here are a couple of cases to get you started:

QA for breakages:

If you've added new elements to the treatment, or modified existing ones, it might be a good idea to check if these new elements are loading as expected across variations, browsers, and devices. Work with your developer to resolve any HTML/CSS breakage issues that might arise.

QA for cross-platform:

Browser incompatibility is one of the most common issues that plagues A/B tests. Check if the treatment is rendering as expected across the most commonly used browsers like Chrome, Safari, Firefox, and Edge. Further, you can run checks across other niche browser choices.

Run a responsiveness check as visitors from mobile devices make for a significant portion of website traffic. 


QA for goal firing:

Suppose you've set a goal that fires only when a visitor from social media spends more than 2 minutes on your blog and subscribes for further updates. QA your A/B test for this exact case. If you've set up Google Analytics integration (or any other third-party integration like Google Analytics, Mixpanel, Kissmetrics) to pass variation-specific data, then check if the the data is flowing seamlessly, and accurately from one platform to the other. 


QA for page load time:

Check the load time of your page before running an A/B test and immediately after you set it up. Generally any difference in page load time is unnoticeable, however, if you see an appreciable increase, then rework the treatment setup. It might be because of the changes you've made to the treatment that are taking a while to load, or even a change in the speed of tracking code snippet's execution.

Note that if you're using a testing tool for performing QA, the load time will be greater than usual—this is not an accurate estimate of the page's actual load time. 

QA for flashing or flickering:

Before making the test live, load the page in your browser to test for any flicker. If you're using an asynchronous code snippet on your website, you may catch a glimpse of the control before the treatment loads. Switch to a synchronous one to give your visitors a smooth website experience. It will hide the control for the short moment it loads and directly show your visitor the treatment. 



Cheers,
Tanya


            • Sticky Posts

            • Did you know?#3: How to target and run your experiment based on custom audience segments in PageSense

              Whatever kind of online business you're running, defining your target audience is the key factor to delivering relevant messages and further positioning your brand in the industry. Most marketing efforts revolve around this basic strategy of segmenting
            • Did you know?#2: How to filter your report by custom date range in PageSense

              Do you want to quickly view your experiment analysis or responses over different time periods? Want to effortlessly compare the performance of your website by specific weeks, months, or years? Also, see whether your new marketing strategies or efforts
            • Did you know?#1: How to filter reports based on different audience segments in PageSense

              Hey everyone, Over recent years, our customer support team has been getting queries asking I have this special requirement, and is it possible to achieve that with PageSense?" Fortunately, they were able to say "Yes" to most of them while providing an
            • [Live Webinar] How to maximize your conversions using existing website traffic

              Hello all,  Did you know that on an average only 0.1% of your traffic converts? The rest are either heading towards your competitors or dropping off without being able to find what they came for. When you want to increase your conversions, it's natural to start by trying to drive more traffic. Of course, you need more traffic for more conversions. But what if we told you, it's wiser to start with the low hanging fruits - your existing visitors. Aptly, our next webinar is titled "How to maximize your
            • [Live Expert Webinar] 7 Deadly Copywriting Sins (and How to Avoid Them)

              Hello, All successful landing pages have one thing in common—powerful words. In other words, good copywriting. At the same time, writing a winning copy that resonates with visitors every single time can be really challenging.   But, here's the catch! Knowing what mistakes to avoid while writing can make the job a lot more easier for you.  And that is why, the next webinar in our CRO Masterminds series will be about "7 Deadly Copywriting Sins (and How to Avoid Them)" . Join us for a free live webinar

            Zoho Pagesense Resources

              Zoho SalesIQ Resources



                    Zoho TeamInbox Resources

                      Zoho DataPrep Resources



                        Zoho CRM Plus Resources

                          Zoho Books Resources


                            Zoho Subscriptions Resources

                              Zoho Desk Resources

                                Zoho Projects Resources


                                  Zoho Sprints Resources


                                    Qntrl Resources


                                      Zoho Creator Resources


                                        Zoho WorkDrive Resources




                                            Zoho Campaigns Resources

                                              Zoho CRM Resources

                                                      Design. Discuss. Deliver.

                                                      Create visually engaging stories with Zoho Show.

                                                      Get Started Now


                                                        Zoho Show Resources