Reassessing Public Opinion Stability

Some characterize public opinion as fickle, volatile, or subject to abrupt change. Yet, empirical evidence from Page and Shapiro (1992) reveals the opposite. This study re-examines the issue of public opinion stability because the six-percentage point criterion Page and Shapiro used ignored the true sample sizes and distributions of many surveys. Calculating the precise statistical test for the difference between two independent proportions shows how smaller opinion changes are potentially statistically significant depending on four parameters: the exact magnitude of the change, the sample size, the distribution of the survey responses, and the design effect correction used. Evenly split (near 50/50) smaller samples (near 1,000 respondents) are generally significant at the six-percentage point level. However, evenly split larger samples (2,000+) attain significance at levels below the six-percentage point threshold, typically around differences of five and four percentage points. In unevenly split distributions (90/10), differences of as little as two percentage points can be statistically significant. Analysis of a subset of The Rational Public data as well as Public Opinion Quarterly Poll Trend data reveals that the metric of opinion stability matters. Many cases previously classified as instances of opinion stability become instances of instability when retaining the sample and distribution characteristics.