Statistical Data Mining of Clinical Data

This chapter provides an introduction into the diverse field of data mining, as viewed from the perspective of a clinical statistician. We start with a discussion of data mining and its relationship with machine learning and classical statistics. To facilitate the presentation of material, we map some common problems occurring in analysis of clinical data onto general machine learning tasks, such as supervised, unsupervised, and semi-supervised learning. We then review key concepts of data mining and machine learning with emphasis on methods that are most relevant for analyses of clinical data. We also present our view of the key elements of a statistical analysis plan that ensure principled data mining of randomized clinical trials. This topic is rarely addressed, yet of interest for many clinical statisticians who are routinely using data mining to gain insights and knowledge from the available data beyond the “pre-specified analyses.” To illustrate the ideas and methods, we provide three case studies based on real and simulated data sets, covering a range of important tasks rarely addressed in common literature on data mining, such as personalized medicine (subgroup identification and dynamic treatment regime optimization) and estimation of treatment effect in the presence of treatment switching. The chapter provides comprehensive up-to-date references to literature on both theory and application of data mining to clinical data, as well as to available software, mostly R packages and SAS procedures.