IDEAS home Printed from https://ideas.repec.org/p/osf/socarx/u3ekq_v1.html

Improving Small-Area Estimates of Public Opinion by Calibrating to Known Population Quantities

Author

Listed:
  • Marble, William
  • Clinton, Joshua

Abstract

Multilevel regression and poststratification is widely used to estimate opinion in small geographies and to adjust unrepresentative surveys — and has become a mainstay in the study of dyadic representation. Yet, errors generated by nonignorable non-response and modeling uncertainty make discrepancies between public opinion and policy outcomes difficult to interpret. We propose a principled, data-driven method to leverage auxiliary quantities with known marginal distributions — e.g., election outcomes — to improve estimates of policy attitudes. Our method estimates the geographic correlation between the auxiliary variables and the outcomes of interest, then adjusts the estimates using observed errors in the auxiliary variable estimate. We illustrate our approach using a pre-election poll measuring support for an abortion referendum, finding that the method reduces county-level error by two-thirds. We find similarly dramatic improvements estimating precinct-level opinion. Our method generates new possibilities for accurately estimating policy attitudes at previously unattainable levels of geographic resolution.

Suggested Citation

  • Marble, William & Clinton, Joshua, 2024. "Improving Small-Area Estimates of Public Opinion by Calibrating to Known Population Quantities," SocArXiv u3ekq_v1, Center for Open Science.
    • Handle: RePEc:osf:socarx:u3ekq_v1
    DOI: 10.31219/osf.io/u3ekq_v1
    as

    Download full text from publisher

    File URL: https://osf.io/download/6658e0d20f8c8002aa3c92cc/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/u3ekq_v1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Rosenman, Evan T. R. & McCartan, Cory & Olivella, Santiago, 2023. "Recalibration of Predicted Probabilities Using the “Logit Shift”: Why Does It Work, and When Can It Be Expected to Work Well?," Political Analysis, Cambridge University Press, vol. 31(4), pages 651-661, October.
    2. Park, David K. & Gelman, Andrew & Bafumi, Joseph, 2004. "Bayesian Multilevel Estimation with Poststratification: State-Level Estimates from National Polls," Political Analysis, Cambridge University Press, vol. 12(4), pages 375-385.
    3. Buttice, Matthew K. & Highton, Benjamin, 2013. "How Does Multilevel Regression and Poststratification Perform with Conventional National Surveys?," Political Analysis, Cambridge University Press, vol. 21(4), pages 449-467.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Roberto Cerina & Raymond Duch, 2021. "Polling India via regression and post-stratification of non-probability online samples," PLOS ONE, Public Library of Science, vol. 16(11), pages 1-34, November.
    2. Cerina, Roberto & Duch, Raymond, 2020. "Measuring public opinion via digital footprints," International Journal of Forecasting, Elsevier, vol. 36(3), pages 987-1002.
    3. Marina Christofoletti & Tânia R. B. Benedetti & Felipe G. Mendes & Humberto M. Carvalho, 2021. "Using Multilevel Regression and Poststratification to Estimate Physical Activity Levels from Health Surveys," IJERPH, MDPI, vol. 18(14), pages 1-16, July.
    4. Wang, Wei & Rothschild, David & Goel, Sharad & Gelman, Andrew, 2015. "Forecasting elections with non-representative polls," International Journal of Forecasting, Elsevier, vol. 31(3), pages 980-991.
    5. Dimiter Toshkov & Elitsa Kortenska, 2015. "Does Immigration Undermine Public Support for Integration in the European Union?," Journal of Common Market Studies, Wiley Blackwell, vol. 53(4), pages 910-925, July.
    6. Kobayashi, Yoshiharu & Heinrich, Tobias & Bryant, Kristin A., 2021. "Public support for development aid during the COVID-19 pandemic," World Development, Elsevier, vol. 138(C).
    7. Lauderdale, Benjamin E. & Bailey, Delia & Blumenau, Jack & Rivers, Douglas, 2020. "Model-based pre-election polling for national and sub-national outcomes in the US and UK," International Journal of Forecasting, Elsevier, vol. 36(2), pages 399-413.
    8. Marina Zaloznaya & William M. Reisinger, 2025. "A Little Too Little, A Little Too Late: The Political Impact of Russia’s Anti-Corruption Enforcement," Laws, MDPI, vol. 14(2), pages 1-19, March.
    9. Temporão, Mickael & Dufresne, Yannick & Savoie, Justin & Linden, Clifton van der, 2019. "Crowdsourcing the vote: New horizons in citizen forecasting," International Journal of Forecasting, Elsevier, vol. 35(1), pages 1-10.
    10. James William Benjamin Elsey & David Manheim & Abigail Marsh & Virginia Schmit & David Moss, 2024. "Ethical acceptability of human challenge trials: Consultation with the US public and with research personnel," PLOS ONE, Public Library of Science, vol. 19(10), pages 1-18, October.
    11. Marcin Hitczenko, 2015. "Identifying and Evaluating Sample Selection Bias in Consumer Payment Surveys," Consumer Payments Research Data Reports 2015-07, Federal Reserve Bank of Atlanta.
    12. Jonathan Gellar & Sarah Hughes & Constance Delannoy & Erin Lipman & Shirley Jeoffreys-Leach & Bobby Berkowitz & Grant J. Robertson, "undated". "Calibrated Multilevel Regression with Poststratifiction for the Analysis of SMS Survey Data," Mathematica Policy Research Reports c71d456bbf9f4026988e1a810, Mathematica Policy Research.
    13. Margaret Weden & Christine Peterson & Jeremy Miles & Regina Shih, 2015. "Evaluating Linearly Interpolated Intercensal Estimates of Demographic and Socioeconomic Characteristics of U.S. Counties and Census Tracts 2001–2009," Population Research and Policy Review, Springer;Southern Demographic Association (SDA), vol. 34(4), pages 541-559, August.
    14. Beth L. Fossen & David A. Schweidel & Michael Lewis, 2019. "Examining Brand Strength of Political Candidates: a Performance Premium Approach," Customer Needs and Solutions, Springer;Institute for Sustainable Innovation and Growth (iSIG), vol. 6(3), pages 63-75, December.
    15. Maurizio Pisati & Valeria Glorioso, 2011. "Multilevel Regression and Poststratification in Stata," CHI11 Stata Conference 4, Stata Users Group.
    16. Beecham, Roger & Clark, Stephen, 2026. "Simulating area-level population outcomes: Should we use Multilevel Regression and Poststratification over Spatial Microsimulation?," SocArXiv erhak_v1, Center for Open Science.
    17. repec:osf:osfxxx:pg2wt_v1 is not listed on IDEAS
    18. Yonatan Ben-Shalom & Ignacio Martinez & Mariel Finucane, "undated". "Risk of Workforce Exit Due to Disability: State Differences in 2003–2016," Mathematica Policy Research Reports 8aed03744a06419dbda68be8c, Mathematica Policy Research.
    19. Caldarulo, Mattia & Mossberger, Karen & Howell, Anthony, 2023. "Community-wide broadband adoption and student academic achievement," Telecommunications Policy, Elsevier, vol. 47(1).
    20. Facchini, Giovanni & Hatton, Timothy J. & Steinhardt, Max F., 2024. "Opening Heaven’s Door: Public Opinion and Congressional Votes on the 1965 Immigration Act," The Journal of Economic History, Cambridge University Press, vol. 84(1), pages 232-270, March.
    21. Benjamin Skinner & Taylor Burtch & Hazel Levy, 2024. "Variation in Broadband Access Among Undergraduate Populations Across the United States," Research in Higher Education, Springer;Association for Institutional Research, vol. 65(5), pages 827-870, August.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:socarx:u3ekq_v1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://arabixiv.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.