Economic Information Acquisition for Improved Predictive Modeling and Decision Making

To better cope with business environments that are information intensive, complex, and highly dynamic, companies increasingly rely on intelligent, data�]driven methods to automatically “learn” from experiences over time and to improve future decision-making. However, reliable data-driven induction depends on two critical ingredients: an effective induction (learning) method, and informative, relevant data.

The capacity of even the most effective induction technique to extract a reliable model of any real-world phenomenon is bounded by what information is available about prior experiences. In practice, organizations often acquire information only passively, such through routine business transactions. However, the opportunity costs of acquiring information only passively can be substantial.

This challenge poses some fundamental and fascinating scientific questions: How does one enable predictive modeling techniques to reason intelligently about opportunities to actively acquire additional information to improve modeling and the business decisions the models informs? How should knowledge (or uncertainty) about the particular decisions that predictive models will be used to inform, influence what information is best to acquire? And how can we incorporate prior knowledge into data-driven learning, so as to mitigate the high cost of learning from scratch? I will present an overview of my research on economic data mining in which I address these and related questions. I will also discuss applications of the methods I developed for a broad range of problems, including personalized marketing, recommender systems, insurance fraud detection, and market mechanism design.