Data Equivalence in Survey Research

When researchers collect data from organizations, groups or individuals operating in apparently different settings with the aim to pool or compare such data, instruments must have similar measurement qualities across such settings. In other words, the data and therefore the measures and the data collection procedures should be equivalent. This paper develops a framework for considering data equivalence during the different stages of survey research. This framework is based on a novel theory of measurement, which distinguishes between a cognitive and a response phase in the response process. The framework provides coherent guidelines for (1) identifying sources of heterogeneity among sub-groups that may threaten equivalence; (2) maximizing equivalence during the design stage of survey research; (3) testing equivalence post data collection and prior to pooling of data; and (4) the management of analytical outcomes. In line with the proposed framework, we review the extent to which equivalence is considered in survey research in six leading empirical Operations and Supply Management (OSM) journals, covering the six-year period from 2006 to 2011. We conclude that pooling of data from apparently heterogeneous sub-groups is common practice in OSM, but that awareness and testing of equivalence remains limited. Our paper emphasizes the need for increased awareness of equivalence issues and provides a clear set of guidelines in order to improve the quality of survey research.