CRO hack #12: QA checks you should conduct before launching your A/B test

CRO hack #12: QA checks you should conduct before launching your A/B test



Prevent spotty user experience by running a thorough quality assurance(QA) check before launching your A/B test. Evaluate the test's behavior under variation-specific edge-cases. Here are a couple of cases to get you started:

QA for breakages:

If you've added new elements to the treatment, or modified existing ones, it might be a good idea to check if these new elements are loading as expected across variations, browsers, and devices. Work with your developer to resolve any HTML/CSS breakage issues that might arise.

QA for cross-platform:

Browser incompatibility is one of the most common issues that plagues A/B tests. Check if the treatment is rendering as expected across the most commonly used browsers like Chrome, Safari, Firefox, and Edge. Further, you can run checks across other niche browser choices.

Run a responsiveness check as visitors from mobile devices make for a significant portion of website traffic. 


QA for goal firing:

Suppose you've set a goal that fires only when a visitor from social media spends more than 2 minutes on your blog and subscribes for further updates. QA your A/B test for this exact case. If you've set up Google Analytics integration (or any other third-party integration like Google Analytics, Mixpanel, Kissmetrics) to pass variation-specific data, then check if the the data is flowing seamlessly, and accurately from one platform to the other. 


QA for page load time:

Check the load time of your page before running an A/B test and immediately after you set it up. Generally any difference in page load time is unnoticeable, however, if you see an appreciable increase, then rework the treatment setup. It might be because of the changes you've made to the treatment that are taking a while to load, or even a change in the speed of tracking code snippet's execution.

Note that if you're using a testing tool for performing QA, the load time will be greater than usual—this is not an accurate estimate of the page's actual load time. 

QA for flashing or flickering:

Before making the test live, load the page in your browser to test for any flicker. If you're using an asynchronous code snippet on your website, you may catch a glimpse of the control before the treatment loads. Switch to a synchronous one to give your visitors a smooth website experience. It will hide the control for the short moment it loads and directly show your visitor the treatment. 



Cheers,
Tanya


    Zoho Developer Community






                              Zoho Desk Resources

                              • Desk Community Learning Series


                              • Digest


                              • Functions


                              • Meetups


                              • Kbase


                              • Resources


                              • Glossary


                              • Desk Marketplace


                              • MVP Corner


                              • Word of the Day



                                  Zoho Marketing Automation


                                          Manage your brands on social media



                                                Zoho TeamInbox Resources

                                                  Zoho DataPrep Resources



                                                    Zoho CRM Plus Resources

                                                      Zoho Books Resources


                                                        Zoho Subscriptions Resources

                                                          Zoho Projects Resources


                                                            Zoho Sprints Resources


                                                              Qntrl Resources


                                                                Zoho Creator Resources


                                                                  Zoho WorkDrive Resources



                                                                    Zoho Campaigns Resources

                                                                      Zoho CRM Resources

                                                                      • CRM Community Learning Series

                                                                        CRM Community Learning Series


                                                                      • Tips

                                                                        Tips

                                                                      • Functions

                                                                        Functions

                                                                      • Meetups

                                                                        Meetups

                                                                      • Kbase

                                                                        Kbase

                                                                      • Resources

                                                                        Resources

                                                                      • Digest

                                                                        Digest

                                                                      • CRM Marketplace

                                                                        CRM Marketplace

                                                                      • MVP Corner

                                                                        MVP Corner







                                                                          Design. Discuss. Deliver.

                                                                          Create visually engaging stories with Zoho Show.

                                                                          Get Started Now