Measuring social norm variation across contexts: Replication and comparison to alternative methods

Studying social norms and how they vary in different contexts requires reliable measurements. We revisit the seminal Krupka and Weber (KW 2013) norm elicitation method and assess the importance of two dimensions of eliciting norms — first, whether to use the KW coordination game method or an alternative, two-stage method that directly elicits first-order or second-order beliefs about social appropriateness, and second, whether to use financial incentives. We replicate KW’s main finding of a qualitative difference in norms between the dictator game and a re-framed version that involves potentially taking money: KW and all other methods show that taking money is less socially appropriate than giving money, holding outcomes fixed and regardless of the presence of monetary incentives. However, we find that the difference in elicited social appropriateness between the two versions of the dictator game varies across methods, with elicited first-order beliefs exhibiting the largest gap in social appropriateness and KW exhibiting the smallest gap. One possible explanation is that strategic uncertainty and complexity in the KW method may attenuate sensitivity of the measure to differences in norms across contexts. A comprehension check reveals that about half of the KW participants initially misunderstood the task, and a prediction exercise reveals that first-order beliefs yield the best predictive power over actual behavior in simple dictator games. One implication is that first-order beliefs could be a simple alternative measure for capturing norm differences across contexts, with good predictive power. A caveat, however, is that first-order beliefs might be more subject to social desirability bias in settings with controversial or pluralistic norms.