Tiny Tip Tuesday #26

Selection Bias in Surveys - I

Selection bias in surveys is when you accidentally create an answer pattern in your survey that users detect and follow, thereby distorting your actual data. How can you fix this? Here are two ways:

Question randomization
Answer randomization

We'll talk about question randomization in this month's tip.

Why should you randomize your survey questions?

As we mentioned earlier, question randomization helps minimize selection bias, which helps you achieve more accurate data, your ultimate goal. So what does question randomization look like?

Question randomization lets you keep the same question order, change the order for each question, or even change the order for a set of selected questions.

Note, however, that randomization doesn't work for questions that have logic or piping applied. This is to avoid tampering with the normal flow and receiving unusual results.

Have you tried out this approach yet? Let us know what else you think could reduce selection bias in surveys.

Watch out for another tip on randomizing answer options.