IDEAS home Printed from https://ideas.repec.org/p/osf/socarx/hp7gz_v1.html

Calibrating Synthetic Respondents: A Within-Persona Experiment on Public Reactions to Protest in Turkey

Author

Listed:
  • KINA, MEHMET FUAT
  • Ekmen, Helin Yaren

Abstract

Large language models are increasingly used to simulate survey respondents, yet the inference pipeline that turns a persona into a response is rarely calibrated for the population and outcome it is applied to. This study makes population- and outcome-specific calibration the research task itself. Using a nationally representative survey in Turkey as the benchmark, we construct persona-conditioned synthetic respondents for all 2,615 survey participants and search a structured space of persona content, sampling strategies, temperatures, model backbones, and prompt framings. Calibration is assessed against two held-out outcomes that carry the response structures on which social movement research typically rests, a rare binary measure of protest participation and a five-point attitude item, and the selected protocols differ between the two. We then run a within-persona synthetic experiment that places the same protest for Kurdish-language education in a political opportunity and a political threat context. The threat context lowers simulated movement legitimacy and willingness to attend, and the simulated contrasts differentiate by political orientation, ethnicity, and education in the directions theory anticipates. Calibration is a precondition for reading synthetic experiments, not a robustness check appended to them.

Suggested Citation

  • KINA, MEHMET FUAT & Ekmen, Helin Yaren, 2026. "Calibrating Synthetic Respondents: A Within-Persona Experiment on Public Reactions to Protest in Turkey," SocArXiv hp7gz_v1, Center for Open Science.
  • Handle: RePEc:osf:socarx:hp7gz_v1
    DOI: 10.31235/osf.io/hp7gz_v1
    as

    Download full text from publisher

    File URL: https://osf.io/download/6ab4fddff9c9ce8ea42c9c1c/
    Download Restriction: no

    File URL: https://libkey.io/10.31235/osf.io/hp7gz_v1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ashwini Ashokkumar & Luke Hewitt & Isaias Ghezae & Robb Willer, 2026. "Large language models can predict the results of social science experiments," Nature, Nature, vol. 656(8126), pages 115-122, August.
    2. Murat Somer, 2019. "Turkey: The Slippery Slope from Reformist to Revolutionary Polarization and Democratic Breakdown," The ANNALS of the American Academy of Political and Social Science, , vol. 681(1), pages 42-61, January.
    3. Young, Lauren E., 2019. "The Psychology of State Repression: Fear and Dissent Decisions in Zimbabwe," American Political Science Review, Cambridge University Press, vol. 113(1), pages 140-155, February.
    4. Argyle, Lisa P. & Busby, Ethan C. & Fulda, Nancy & Gubler, Joshua R. & Rytting, Christopher & Wingate, David, 2023. "Out of One, Many: Using Language Models to Simulate Human Samples," Political Analysis, Cambridge University Press, vol. 31(3), pages 337-351, July.
    5. Edwards, Pearce & Arnon, Daniel, 2021. "Violence on Many Sides: Framing Effects on Protest and Support for Repression," British Journal of Political Science, Cambridge University Press, vol. 51(2), pages 488-506, April.
    6. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769, December.
    7. Bisbee, James & Clinton, Joshua D. & Dorff, Cassy & Kenkel, Brenton & Larson, Jennifer M., 2024. "Synthetic Replacements for Human Survey Data? The Perils of Large Language Models," Political Analysis, Cambridge University Press, vol. 32(4), pages 401-416, October.
    8. Balcells, Laia & Dorsey, Spencer & Tellez, Juan F., 2021. "Repression and Dissent in Contemporary Catalonia," British Journal of Political Science, Cambridge University Press, vol. 51(4), pages 1742-1750, October.
    9. Antonina Rafikova & Anatoly Voronin, 2026. "ChatGPT as a research proxy: simulating human attitudes in social science research," Journal of Computational Social Science, Springer, vol. 9(1), pages 1-30, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Matthew O. Jackson & Benjamin S. Manning & Yutong Xie & Walter Yuan & Qiaozhu Mei, 2026. "Modeling Human Behavior with Type Vectors Using AI," Papers 2608.18265, arXiv.org, revised Sep 2026.
    2. Gabriele Ruiu & Giovanna Gonano, 2020. "Religious Barriers to the Diffusion of Same-sex Civil Unions in Italy," Population Research and Policy Review, Springer;Southern Demographic Association (SDA), vol. 39(6), pages 1185-1203, December.
    3. Wright, Austin L. & Sonin, Konstantin & Driscoll, Jesse & Wilson, Jarnickae, 2020. "Poverty and economic dislocation reduce compliance with COVID-19 shelter-in-place protocols," Journal of Economic Behavior & Organization, Elsevier, vol. 180(C), pages 544-554.
    4. Guido de Blasio & Daniela Vuri, 2019. "Effects of the Joint Custody Law in Italy," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 16(3), pages 479-514, September.
    5. Graves Jennifer & McMullen Steven & Rouse Kathryn, 2018. "Teacher Turnover, Composition and Qualifications in the Year-Round School Setting," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 18(3), pages 1-27, July.
    6. Alston Lee J. & Mueller Bernardo, 2018. "Priests, Conflicts and Property Rights: the Impacts on Tenancy and Land Use in Brazil," Man and the Economy, De Gruyter, vol. 5(1), pages 1-26, June.
    7. S Anukriti & Catalina Herrera‐Almanza & Praveen K. Pathak & Mahesh Karra, 2020. "Curse of the Mummy‐ji: The Influence of Mothers‐in‐Law on Women in India†," American Journal of Agricultural Economics, John Wiley & Sons, vol. 102(5), pages 1328-1351, October.
    8. Ellison, Richard B. & Ellison, Adrian B. & Greaves, Stephen P. & Sampaio, Breno, 2017. "Electronic ticketing systems as a mechanism for travel behaviour change? Evidence from Sydney’s Opal card," Transportation Research Part A: Policy and Practice, Elsevier, vol. 99(C), pages 80-93.
    9. Cruzatti C., John & Bjørnskov, Christian & Sáenz de Viteri, Andrea & Cruzatti, Christian, 2024. "Geography, development, and power: Parliament leaders and local clientelism," World Development, Elsevier, vol. 182(C).
    10. Yusuke Matsuki, 2016. "A Distribution-Free Test of Monotonicity with an Application to Auctions," Working Papers e110, Tokyo Center for Economic Research.
    11. Shvartsman, Elena & Beckmann, Michael, 2015. "Stressed by your job: What is the role of personnel policy?," Working papers 2015/15, Faculty of Business and Economics - University of Basel.
    12. Eric Hitz & Mingmin Feng & Radu Tanase & Ren'e Algesheimer & Manuel S. Mariani, 2025. "The amplifier effect of artificial agents in social contagion," Papers 2502.21037, arXiv.org, revised Mar 2025.
    13. MacDonald, Peter, 2013. "Labour substitution and the scope for military outsourcing," MPRA Paper 46688, University Library of Munich, Germany.
    14. Nicholas Sim, 2015. "Astronomics In Action: The Graduate Earnings Premium And The Dragon Effect In Singapore," Economic Inquiry, Western Economic Association International, vol. 53(2), pages 922-939, April.
    15. Matteo Migheli, 2021. "Green purchasing: the effect of parenthood and gender," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 23(7), pages 10576-10600, July.
    16. Yuyang Li & Jiahui Li & Xinjie Li & Qian Lu, 2024. "Does Participation in Digital Supply and Marketing Promote Smallholder Farmers’ Adoption of Green Agricultural Production Technologies?," Land, MDPI, vol. 14(1), pages 1-24, December.
    17. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    18. Sondang Marsinta Uli Panggabean & Mahjus Ekananda & Beta Yulianita Gitaharie & Leslie Djuranovik, 2025. "Export proceeds repatriation policies: A shield against exchange rate volatility in emerging markets?," Papers 2506.09168, arXiv.org.
    19. Laurent Didier, 2017. "South-South Trade and Geographical Diversification of Intra-SSA Trade: Evidence from BRICs," African Development Review, African Development Bank, vol. 29(2), pages 139-154, June.
    20. Han, Luyi & Winters, John & Betz, Michael, 2026. "Do Prior Residents Benefit from Energy Booms?," IZA Discussion Papers 18466, IZA Network @ LISER.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:socarx:hp7gz_v1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://socarxiv.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.