SPLIT TEST Student: Khudaybergenov T.A. Professor: Kwon You Youn What Is a Split Test? • A split test (or A/B testing) is a method of testing multiple versions of a web site (or individual elements of a website) against each other to see which works best. • Ideally, there will be only one difference between the two pages so the tester can understand the reason behind the change in performance. Split Test • By comparing several versions of your web pages, such as your landing pages or homepages, a split test helps you identify which one has a better conversion rate for your visitors. • When the split test is launched, your pages’ traffic is randomly spread over the different versions of your pages. Each one’s performance is tracked and analyzed by the split testing software to identify the version that converts the best, with the highest significance. Split testing determines the version on which the sample converted the best. Why Should We Run a Split Test? 1. 2. 3. 4. 5. 6. 7. Getting rid of any uncertainty about marketing campaigns by eliminating guesswork. Learning how customers actually respond, rather than how they think they will respond, which can happen with survey data. Gaining unexpected insights that can drive improvements across the company. Creating better and more effective content. Maximizing web traffic and visitor engagement. Reducing risk by testing changes before making them permanent. Improving revenue. If you get more leads and sales by making small changes, then your business becomes more profitable. Overall, split testing is a cost-effective and easy way to improve your website and marketing. “If I get 100 people to my site, and I have a 20% conversion rate, that means I get 20 people to convert... I can try to get that conversion rate to 35% and get 35 people to convert, or, I could just figure out how to get 1,000 new visitors, maintain that 20% conversion, and you’ll see that 20% of 1,000 (200), is much higher than 35% of 100 (35).” Derek Halpern - SocialTriggers.com What Can You Test? Almost anything on your website that affects visitor behavior can be A/B tested. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. Headlines Sub headlines Paragraph Text Testimonials Call to Action text Call to Action Button Links Images Content near the fold Social proof Media mentions Awards and badges A/B Testing Process Study your Website Data: Use a website analytics tool such as Google Analytics, and 1. find the problem areas in your conversion funnel. For example, you can identify the pages with the highest bounce rate. Let's say, your homepage has an unusually high bounce rate. 2. Observe User Behavior: Utilize visitor behavior analysis tools such as Heatmaps, Visitor Recordings, Form Analysis and On-page Surveys, and find what is stopping the visitors from converting. For example, “The CTA button is not prominent on the home page." 3. Construct a Hypothesis: Per the insights from visitor behavior analysis tools, build a hypothesis aimed at increasing conversions. For example, “Increasing the size of the CTA button will make it more prominent and will increase conversions.” 4. Test your Hypothesis: Create a variation per your hypothesis, and A/B test it against the original page. For example, “A/B test your original home page against a version that has a larger CTA button.” Calculate the test duration with respect to the number of your monthly visitors, current conversion rate, and the expected change in the conversion rate. (Use our Bayesian Calculator here.) 5. Analyze Test Data and Draw Conclusions: Analyze the A/B test results, and see which variation delivered the highest conversions. If there is a clear winner among the variations, go ahead with its implementation. If the test remains inconclusive, go back to step number three and rework your hypothesis. 6. Report results to all concerned: Let others in Marketing, IT, and UI/UX know of the test results and the insights generated. Options • Most A/B/n tests work with two to four variations. Mistakes to Avoid With Split Testing • Testing the wrong pages. • Testing too many variations • Not forming a proper hypothesis before start • Timing • Running a split test at the wrong time Multivariate testing • Full factorial testing, which tests every possible combination of elements until there’s a clear winner. This needs a lot of traffic, and traffic is distributed evenly among the variations. • Fractional factorial testing (often using the Taguchi Method), which uses a sampling method to test combinations and statistical analysis to decide on the winner. However, this means you are partially relying on assumptions, rather than data. • Adaptive testing, which uses live data on visitor actions to decide on the winning combination. 10 Split Testing Tools • CrazyEgg • HotJar • Optimizely • Unbounce • Visual Website Optimizer Word press • ABPress Optimizer • Nelio • Simple Page Tester • Title Experiments (here’s a tutorial) • WordPress Calls to Action References • https://www.omniconvert.com/what-is/split-testing/ • https://vwo.com/ab-testing/ • https://www.abtasty.com/split-testing/ • https://instapage.com/blog/what-is-split-testing • https://optinmonster.com/conversion-optimization-101- split-testing-vs-multivariate-testing/ • https://optinmonster.com/how-to-create-a-split-test-andwhy-you-should/ Thank you! 깜사합니다!
