Leveraging Repeated Marketing Interventions for Effective Targeting/Personalization

Targeting customers is at the heart of marketing strategy. To do so effectively, firms need to understand the effectiveness of their targeting efforts across customers, over time, and for different levers that are at the firm’s disposal. Recent advances in data collection, analyses, and technology have fueled the ability of firms to personalize their marketing efforts. To learn which marketing actions are likely to generate the most positive response, marketers often run A/B tests. Researchers have armed firms with tools that leverage experimentation to estimate heterogeneous treatment effects in order to inform personalized targeting. These approaches often use machine learning (ML) tools to relate customer characteristics observed prior to the intervention to the magnitude of the impact of the intervention. Firms can then use these customer characteristics to select targets for whom the marketing intervention is likely to results in stronger effects.

However, while these approaches provide estimates of heterogeneous treatment effects with respect to the observed intervention(s), they often fail to quantify how much of the treatment effect is due to (1) the design of the intervention, (2) the customer’s sensitivity to interventions, and (3) contextual factors such as time of the day or day of the week. Furthermore, these one-shot approaches fail to recognize the fact that the same customer is often exposed to multiple interventions over time. As a result, the insights may not be fully generalizable to interventions conducted in a different context, on a different date, or with a somewhat different design.

The objective of this research is twofold. First, we explore the heterogeneity and dynamics of marketing interventions in the context of field experiments, decomposing the effectiveness of the firm’s marketing actions into aspects related to the design of the intervention, the heterogeneity in susceptibility of customers to marketing actions, and other contextual factors. Second, we propose a modeling framework that is simple, scalable and that leverages customers’ exposure to multiple treatments over time. More specifically, we develop a hierarchical Bayesian shrinkage approach to model the treatment effect of marketing interventions as a function of observed and unobserved campaign characteristics, time, and unobserved customer-level heterogeneous effects.

For the empirical application, we observe data on nearly 3 million users who, over the course of 2.5 years, were exposed to approximately 2,000 randomized marketing interventions (A/B tests), with an average of over 40 interventions per user. The repeated exposure to interventions allows us to study how much of the treatment effect variation is due to campaign characteristics (e.g., incentives with an educational goal versus incentives to prevent churn), individual sensitivity to marketing actions, and contextual effects (such as when the promotion was run). Furthermore, we can investigate whether there are signs of ‘over-targeting’, that is, whether marketing campaigns become less effective once customers have been exposed to interventions in the past.