Perceived Accuracy of Survey Research: The Roles of “n” and “n/N”

From a statistical point of view the accuracy of survey results—for finite populations—is a function of both sample size and the ratio of sample-to-population size. Holding constant population size, the standard error of an estimate for a variable of interest is smaller for larger samples. Holding constant sample size, the standard error of an estimate for a variable of interest is smaller for larger sample-to-population ratios. Both relationships are highly nonlinear. We designed six studies to examine people’s lay beliefs. We find that people take both cues into account, but that they weight the sample-to-population ratio much more than the sample size, even though objectively sample size matters more. People also tend to linearize the relationships between these cues and standard error, especially in joint evaluation mode. These incorrect beliefs may lead researchers and managers to allocate resources in a suboptimal way.