How to Interpret Results of Meta-Analysis

Introduction

Meta-analysis is a systematic method for synthesizing quantitative results of different empirical studies regarding the effect of an independent variable (or determinant, or intervention, or treatment) on a defined outcome (or dependent variable). Mainly developed in medical and psychological research as a tool for synthesizing empirical information about the outcomes of a treatment, meta-analysis is now increasingly used in the social sciences as a tool for hypothesis testing. However, the assumptions underlying meta-analytic hypothesis testing in the social sciences will usually not be met under real-life conditions. This is the reason why meta-analysis is increasingly conducted with a different aim, based on more realistic assumptions. That aim is to explore the dispersion of effect sizes.

Structure

The structure of this guide follows the structure of the workbooks of Meta-Essentials:

1. Introduction
   1.1 Assumptions
2. The forest plot
   2.1 Confidence interval: hypothesis testing
   2.2 Estimating the extent of heterogeneity
   2.3 Prediction interval
   2.4 Model
3. Subgroup analysis
4. Moderator analysis
5. Publication bias analysis
6. Conclusion

Assuming that the appropriate workbook has been chosen and that the relevant information about the different studies has been entered on the Input sheet of that workbook, it discusses the interpretation of the forest plot, subgroup analysis, moderator analysis, and publication bias analyses.

Aim of this guide

The aim of this guide is to support the researcher in interpreting the results of a meta-analysis. Throughout this guide we will use statistics, figures, and tables as provided by Meta-Essentials, user-friendly software for meta-analysis that is freely downloadable (http://www.erim.eur.nl/research-support/meta-essentials/). Although the figures and tables in this document are taken from examples in Meta-Essentials, its contents are applicable to any meta-analysis irrespective of the software package that is used.

When the Meta-Essentials software is downloaded from the website, its workbooks are already filled with (fictional) data. The examples and screen-prints that are used in the user manual are taken from Workbook 1 (Effect size data.xls). The same examples and screen-prints are used in this document. This means that the reader can generate and manipulate these same examples in that workbook, which might be useful for a critical reading of the following text.