Guessing and Forgetting: A Latent Class Model for Measuring Learning

Guessing on closed-ended knowledge items is common. Under likely-to-hold assumptions, in the presence of guessing, the most common estimator of learning, difference between pre- and postprocess scores, is negatively biased. To account for guessing-related error, we develop a latent class model of how people respond to knowledge questions and identify the model with the mild assumption that people do not lose knowledge over short periods of time. A Monte Carlo simulation over a broad range of informative processes and knowledge items shows that the simple difference score is negatively biased and the method we develop here is unbiased. To demonstrate its use, we apply our model to data from Deliberative Polls. We find that estimates of learning, once adjusted for guessing, are about 13% higher. Adjusting for guessing also eliminates the gender gap in learning, and halves the pre-deliberation gender gap on political knowledge.