![]() The increase in reliability and validity by adding more points to a scale doesn’t go on forever and it isn’t a linear increase. We pull much of this from the work of Jum Nunnally, the noted psychometrician who wrote about the reliability of scales in 1967, 1978 and in 1994. The bluntness of scales with few points reduces correlations and causes the reduction in reliability and validity. The number of scale points is a topic we address in our UX Measurement Bootcamp and Jim Lewis and I have discussed it more extensively in Chapter 9 of Quantifying the User Experience.Īs a general rule, when measuring a construct that falls on a continuum from low to high (such as satisfaction, ease, and likelihood to recommend), the more points you have in your rating scale, the more reliable (consistent responses) and valid (reflects true attitudes) it generally is. And a good place to start is the published literature that contains a wealth of research. While it’s good to start with theories of why one response scale would be better than another (for example, the theory that shorter scales are easier and better), it’s important to verify or falsify those claims with data. ![]() The references often just point to each other and offer at best a sort of folk wisdom and rationale, sort of like “ all content should be reachable in three clicks”-because why make users click more? Unfortunately, these articles offer no justification for the position. Responses collected from a large, 11-point scale are extremely noisy, and meaningful changes in ratings are hard to detect … A better scale would use a 3-option Yes/Maybe/No system or a similar scale with 5 options. This advice gets repeated by others on the Internet: If you measure very specific experiences, it is very easy for a user to decide if they are happy or not. If you feel strongly about a 4 or 5-point scale, go for it. This sentiment is echoed in another post, this time about ten-point scales, which apparently “don’t reflect reality.”Īnd this one from Tomer who also advocates for the simplicity of two- and three-point scales: Eleven points “pretend noise is science.”įew things seem to elicit more opinions (and misinformation) in measurement than the “right” number of scale points to use in a rating scale response option.įor example, here is a discussion on Twitter by Erika Hall making the case that eleven-point scales are difficult for humans to respond to and three points would be better.
0 Comments
Leave a Reply. |