IDEAS home Printed from https://ideas.repec.org/p/osf/socarx/u3ekq_v1.html

Improving Small-Area Estimates of Public Opinion by Calibrating to Known Population Quantities

Author

Listed:
  • Marble, William
  • Clinton, Joshua

Abstract

Multilevel regression and poststratification is widely used to estimate opinion in small geographies and to adjust unrepresentative surveys — and has become a mainstay in the study of dyadic representation. Yet, errors generated by nonignorable non-response and modeling uncertainty make discrepancies between public opinion and policy outcomes difficult to interpret. We propose a principled, data-driven method to leverage auxiliary quantities with known marginal distributions — e.g., election outcomes — to improve estimates of policy attitudes. Our method estimates the geographic correlation between the auxiliary variables and the outcomes of interest, then adjusts the estimates using observed errors in the auxiliary variable estimate. We illustrate our approach using a pre-election poll measuring support for an abortion referendum, finding that the method reduces county-level error by two-thirds. We find similarly dramatic improvements estimating precinct-level opinion. Our method generates new possibilities for accurately estimating policy attitudes at previously unattainable levels of geographic resolution.

Suggested Citation

  • Marble, William & Clinton, Joshua, 2024. "Improving Small-Area Estimates of Public Opinion by Calibrating to Known Population Quantities," SocArXiv u3ekq_v1, Center for Open Science.
    • Handle: RePEc:osf:socarx:u3ekq_v1
    DOI: 10.31219/osf.io/u3ekq_v1
    as

    Download full text from publisher

    File URL: https://osf.io/download/6658e0d20f8c8002aa3c92cc/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/u3ekq_v1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Rosenman, Evan T. R. & McCartan, Cory & Olivella, Santiago, 2023. "Recalibration of Predicted Probabilities Using the “Logit Shift”: Why Does It Work, and When Can It Be Expected to Work Well?," Political Analysis, Cambridge University Press, vol. 31(4), pages 651-661, October.
    2. Park, David K. & Gelman, Andrew & Bafumi, Joseph, 2004. "Bayesian Multilevel Estimation with Poststratification: State-Level Estimates from National Polls," Political Analysis, Cambridge University Press, vol. 12(4), pages 375-385.
    3. Buttice, Matthew K. & Highton, Benjamin, 2013. "How Does Multilevel Regression and Poststratification Perform with Conventional National Surveys?," Political Analysis, Cambridge University Press, vol. 21(4), pages 449-467.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Roberto Cerina & Raymond Duch, 2021. "Polling India via regression and post-stratification of non-probability online samples," PLOS ONE, Public Library of Science, vol. 16(11), pages 1-34, November.
    2. Cerina, Roberto & Duch, Raymond, 2020. "Measuring public opinion via digital footprints," International Journal of Forecasting, Elsevier, vol. 36(3), pages 987-1002.
    3. Marina Christofoletti & Tânia R. B. Benedetti & Felipe G. Mendes & Humberto M. Carvalho, 2021. "Using Multilevel Regression and Poststratification to Estimate Physical Activity Levels from Health Surveys," IJERPH, MDPI, vol. 18(14), pages 1-16, July.
    4. Wang, Wei & Rothschild, David & Goel, Sharad & Gelman, Andrew, 2015. "Forecasting elections with non-representative polls," International Journal of Forecasting, Elsevier, vol. 31(3), pages 980-991.
    5. Kobayashi, Yoshiharu & Heinrich, Tobias & Bryant, Kristin A., 2021. "Public support for development aid during the COVID-19 pandemic," World Development, Elsevier, vol. 138(C).
    6. Lauderdale, Benjamin E. & Bailey, Delia & Blumenau, Jack & Rivers, Douglas, 2020. "Model-based pre-election polling for national and sub-national outcomes in the US and UK," International Journal of Forecasting, Elsevier, vol. 36(2), pages 399-413.
    7. Marina Zaloznaya & William M. Reisinger, 2025. "A Little Too Little, A Little Too Late: The Political Impact of Russia’s Anti-Corruption Enforcement," Laws, MDPI, vol. 14(2), pages 1-19, March.
    8. Jonathan Gellar & Sarah Hughes & Constance Delannoy & Erin Lipman & Shirley Jeoffreys-Leach & Bobby Berkowitz & Grant J. Robertson, "undated". "Calibrated Multilevel Regression with Poststratifiction for the Analysis of SMS Survey Data," Mathematica Policy Research Reports c71d456bbf9f4026988e1a810, Mathematica Policy Research.
    9. Margaret Weden & Christine Peterson & Jeremy Miles & Regina Shih, 2015. "Evaluating Linearly Interpolated Intercensal Estimates of Demographic and Socioeconomic Characteristics of U.S. Counties and Census Tracts 2001–2009," Population Research and Policy Review, Springer;Southern Demographic Association (SDA), vol. 34(4), pages 541-559, August.
    10. Maurizio Pisati & Valeria Glorioso, 2011. "Multilevel Regression and Poststratification in Stata," CHI11 Stata Conference 4, Stata Users Group.
    11. Beecham, Roger & Clark, Stephen, 2026. "Simulating area-level population outcomes: Should we use Multilevel Regression and Poststratification over Spatial Microsimulation?," SocArXiv erhak_v1, Center for Open Science.
    12. repec:osf:osfxxx:pg2wt_v1 is not listed on IDEAS
    13. Caldarulo, Mattia & Mossberger, Karen & Howell, Anthony, 2023. "Community-wide broadband adoption and student academic achievement," Telecommunications Policy, Elsevier, vol. 47(1).
    14. Benjamin Skinner & Taylor Burtch & Hazel Levy, 2024. "Variation in Broadband Access Among Undergraduate Populations Across the United States," Research in Higher Education, Springer;Association for Institutional Research, vol. 65(5), pages 827-870, August.
    15. Thuy Nguyen & Victoria Perez, 2020. "Privatizing Plaintiffs: How Medicaid, the False Claims Act, and Decentralized Fraud Detection Affect Public Fraud Enforcement Efforts," Journal of Risk & Insurance, The American Risk and Insurance Association, vol. 87(4), pages 1063-1091, December.
    16. Christopher Claassen & Richard Traunmüller, 2020. "Improving and Validating Survey Estimates of Religious Demography Using Bayesian Multilevel Models and Poststratification," Sociological Methods & Research, , vol. 49(3), pages 603-636, August.
    17. Tingzhong Yang & Dan Wu & Weifang Zhang & Randall R Cottrell & Ian R H Rockett, 2012. "Comparative Stress Levels among Residents in Three Chinese Provincial Capitals, 2001 and 2008," PLOS ONE, Public Library of Science, vol. 7(11), pages 1-6, November.
    18. Agnolin, Paolo & Anelli, Massimo & Colantone, Italo & Stanig, Piero, 2025. "Robots Replacing Trade Unions: Novel Data and Evidence from Western Europe," IZA Discussion Papers 17864, IZA Network @ LISER.
    19. Jinan N. Allan & Joseph T. Ripberger & Wesley Wehde & Makenzie Krocak & Carol L. Silva & Hank C. Jenkins‐Smith, 2020. "Geographic Distributions of Extreme Weather Risk Perceptions in the United States," Risk Analysis, John Wiley & Sons, vol. 40(12), pages 2498-2508, December.
    20. Donghui Wang & Yongai Jin & Tao Liu, 2025. "Universal yet local: Estimating county-level fertility ideals and intentions in China," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 53(18), pages 525-568.
    21. Bullinger, Lindsey Rose & Wing, Coady, 2019. "How many children live with adults with opioid use disorder?," Children and Youth Services Review, Elsevier, vol. 104(C), pages 1-1.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:socarx:u3ekq_v1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://arabixiv.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.