Effect sizes of the d-family
The workbooks and a pdf-version of this user manual can be downloaded from here.
Research designs in the d-family can be categorized along two dimensions:
- The dependent variable can be categorical or continuous. This difference results in different types of effect size, namely a difference between proportions if the dependent variable is categorical and a difference between means if the dependent variable is continuous.
- The difference that is studied can be a difference between different groups or a within-group difference. Examples of the first type (“independent groups”) are experiments with separate groups and non-experimental differences between separate categories (e.g., between men and women, or between different types of companies). An example of the second type (“dependent groups”) is a difference in time, for instance before and after a therapy or other intervention.
Four types of studies with a d-design can be distinguished based on these two dimensions (see Table 2). Workbooks 2, 3 and 4 each fill one of the cells in table. The cell for categorical dependent variable with dependent groups is empty because this type of design is very rare. Should you want to meta-analyse effect sizes of such type you can use workbook 1 ‘Effect size data.xlsx’.
| Independent groups | Dependent groups | |
|---|---|---|
| Categorical dependent variable | 2. Differences between independent groups - binary data.xlsx | |
| Numerical dependent variable | 3. Differences between independent groups - continuous data.xlsx | 4. Differences between dependent groups - continuous data.xlsx |
Table 2: Overview of the Meta-Essentials workbooks of the d-family
Workbook 2 ‘Differences between independent groups - binary data.xlsx’ can be used for meta-analysing studies that compare two groups (typically an experimental group and a control group) when the outcome of interest is categorical (e.g., success versus failure). This is a common research design in clinical studies but could be applied in social sciences as well. For instance, the relationship of the gender of an entrepreneur with the one-year survival (survival versus bankruptcy) of a start-up could in one study be evaluated with a two-by-two table. Typical statistics to grasp the size of difference in such studies are the odds ratio, risk ratio, and the risk difference.
Workbook 3 ‘Differences between independent groups - continuous data.xlsx’ is designed to meta-analyse studies of which the outcome is a difference between the means of two independent groups. For instance, to test whether a training has a positive effect on the sales of sales personnel, a study might be designed that gives one group of salespersons a training and another group no training. The effect size of interest would then be the difference between the average sales of the persons that received training compared to that of the persons that did not receive training.
Workbook 4 ‘Differences between dependent groups - continuous data.xlsx’ is designed to meta-analyse studies of which the outcome is a difference between the means of two measurements in the same group. In comparison to the previous example, this is the effect size in a study of a difference in sales in the same group of persons before and after training. This is often referred to as a pre-posttest study design. On the face of it, there are few differences between workbooks 3 and 4. However, the calculations ’behind’ the workbooks are different.
