IDEAS home Printed from https://ideas.repec.org/a/eee/beexfi/v48y2025ics2214635025000991.html

Conducting real-time interactive experiments online: A guide for researchers

Author

Listed:
  • Flecke, Sarah Lynn
  • Bachler, Sebastian

Abstract

The COVID-19 pandemic has accelerated the shift towards online behavioral experiments and the increased adoption of crowdsourcing platforms for participant recruitment. While conducting online studies has become more common, carrying out real-time interactive experiments online presents unique challenges. We provide an overview of existing tools for conducting interactive online experiments and discuss key considerations for setting up and running interactive online studies. We make special mention of the setup combination of studies programmed in oTree and carried out with Prolific samples, which has become popular in experimental and behavioral economics and finance research. Using a case study of a large multi-round experiment involving synchronous decision-making among groups of participants recruited via Prolific, we examine critical factors such as sample availability, participant arrival speed, traffic management, as well as dropout risks and mitigation approaches. We discuss strategies researchers can adopt and suggest areas for further investigation.

Suggested Citation

  • Flecke, Sarah Lynn & Bachler, Sebastian, 2025. "Conducting real-time interactive experiments online: A guide for researchers," Journal of Behavioral and Experimental Finance, Elsevier, vol. 48(C).
    • Handle: RePEc:eee:beexfi:v:48:y:2025:i:c:s2214635025000991
    DOI: 10.1016/j.jbef.2025.101118
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S2214635025000991
    Download Restriction: no
    File URL: https://libkey.io/10.1016/j.jbef.2025.101118?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. John Horton & David Rand & Richard Zeckhauser, 2011. "The online laboratory: conducting experiments in a real labor market," Experimental Economics, Springer;Economic Science Association, vol. 14(3), pages 399-425, September.
    2. Barbara Ikica & Simon Jantschgi & Heinrich H Nax & Diego G Nuñez Duran & Bary S R Pradelski, 2023. "Competitive market behavior: convergence and asymmetry in the experimental double auction," Post-Print hal-04367512, HAL.
    3. Goeschl, Timo & Haberl, Beatrix & Soldà, Alice, 2023. "How to Organize Monitoring and Punishment: Experimental Evidence," Working Papers 0737, University of Heidelberg, Department of Economics.
    4. Chan, Nathan W. & Wolk, Leonard, 2020. "Cost-effective giving with multiple public goods," Journal of Economic Behavior & Organization, Elsevier, vol. 173(C), pages 130-145.
    5. Paolacci, Gabriele & Chandler, Jesse & Ipeirotis, Panagiotis G., 2010. "Running experiments on Amazon Mechanical Turk," Judgment and Decision Making, Cambridge University Press, vol. 5(5), pages 411-419, August.
    6. Irene Maria Buso & Daniela Di Cagno & Lorenzo Ferrari & Vittorio Larocca & Luisa Lorè & Francesca Marazzi & Luca Panaccione & Lorenzo Spadoni, 2021. "Lab-like findings from online experiments," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 7(2), pages 184-193, December.
    7. James Pettit & Daniel Friedman & Curtis Kephart & Ryan Oprea, 2014. "Software for continuous game experiments," Experimental Economics, Springer;Economic Science Association, vol. 17(4), pages 631-648, December.
    8. Huber, Christoph & Huber, Jürgen, 2020. "Bad bankers no more? Truth-telling and (dis)honesty in the finance industry," Journal of Economic Behavior & Organization, Elsevier, vol. 180(C), pages 472-493.
    9. Hyndman, Kyle & Walker, Matthew J., 2022. "Fairness and risk in ultimatum bargaining," Games and Economic Behavior, Elsevier, vol. 132(C), pages 90-105.
    10. Wladislaw Mill & John Morgan, 2022. "The cost of a divided America: an experimental study into destructive behavior," Experimental Economics, Springer;Economic Science Association, vol. 25(3), pages 974-1001, June.
    11. Neeraja Gupta & Luca Rigotti & Alistair Wilson, 2021. "The Experimenters' Dilemma: Inferential Preferences over Populations," Papers 2107.05064, arXiv.org, revised Jul 2021.
    12. Barbara Ikica & Simon Jantschgi & Heinrich H. Nax & Diego G. Nuñez Duran & Bary S. R. Pradelski, 2023. "Competitive Market Behavior: Convergence And Asymmetry In The Experimental Double Auction," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 64(3), pages 1087-1126, August.
    13. Gabriele Paolacci & Jesse Chandler & Panagiotis G. Ipeirotis, 2010. "Running experiments on Amazon Mechanical Turk," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 5(5), pages 411-419, August.
    14. Ofra Amir & David G Rand & Ya'akov Kobi Gal, 2012. "Economic Games on the Internet: The Effect of $1 Stakes," PLOS ONE, Public Library of Science, vol. 7(2), pages 1-4, February.
    15. Bajoori, Elnaz & Peeters, Ronald & Wolk, Leonard, 2024. "Security auctions with cash- and equity-bids: An experimental study," European Economic Review, Elsevier, vol. 163(C).
    16. Molnar, Andras, 2019. "SMARTRIQS: A Simple Method Allowing Real-Time Respondent Interaction in Qualtrics Surveys," Journal of Behavioral and Experimental Finance, Elsevier, vol. 22(C), pages 161-169.
    17. Huber, Christoph & Litsios, Christos & Nieper, Annika & Promann, Timo, 2023. "On social norms and observability in (dis)honest behavior," Journal of Economic Behavior & Organization, Elsevier, vol. 212(C), pages 1086-1099.
    18. Marcus Giamattei & Kyanoush Seyed Yahosseini & Simon Gächter & Lucas Molleman, 2020. "LIONESS Lab: a free web-based platform for conducting interactive experiments online," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 6(1), pages 95-111, June.
    19. Greiff, Matthias & Giamattei, Marcus, 2024. "Spillovers from incentive schemes on distributional preferences and expectations," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 112(C).
    20. Joseph Henrich & Steve J. Heine & Ara Norenzayan, 2010. "The Weirdest People in the World?," RatSWD Working Papers 139, German Data Forum (RatSWD).
    21. Malthouse, Eugene & Pilgrim, Charlie & Hills, Thomas & Sgroi, Daniel, 2023. "When Fairness Is Not Enough: The Disproportionate Contributions of the Poor in a Collective Action Problem," IZA Discussion Papers 16179, IZA Network @ LISER.
    22. Bilancini, Ennio & Boncinelli, Leonardo & Capraro, Valerio & Celadin, Tatiana & Paolo, Roberto Di, 2020. "“Do the right thing” for whom? An experiment on ingroup favouritism, group assorting and moral suasion," Judgment and Decision Making, Cambridge University Press, vol. 15(2), pages 182-192, March.
    23. Nathan W. Chan & Leonard Wolk, 2023. "Reciprocity with stochastic loss," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 9(1), pages 51-65, June.
    24. Burgstaller, Lilith & Pfeil, Katharina, 2024. "You don’t need an invoice, do you? An online experiment on collaborative tax evasion," Journal of Economic Psychology, Elsevier, vol. 101(C).
    25. repec:osf:osfxxx:b5682_v1 is not listed on IDEAS
    26. Chapkovski, Philipp, 2023. "Conducting interactive experiments on Toloka," Journal of Behavioral and Experimental Finance, Elsevier, vol. 37(C).
    27. Capraro, Valerio & Rodriguez-Lara, Ismael & Ruiz-Martos, Maria J., 2020. "Preferences for efficiency, rather than preferences for morality, drive cooperation in the one-shot Stag-Hunt game," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 86(C).
    28. Mill, Wladislaw & Morgan, John, 2022. "Competition between friends and foes," European Economic Review, Elsevier, vol. 147(C).
    29. Ennio Bilancini & Leonardo Boncinelli & Valerio Capraro & Tatiana Celadin & Roberto Di Paolo, 2020. ""Do the right thing" for whom? An experiment on ingroup favouritism, group assorting and moral suasion," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 15(2), pages 182-192, March.
    30. Milosav, Đorđe & Nistotskaya, Marina, 2024. "Unpacking the relevance of interpersonal trust in the blockchain era: Theory and experimental evidence," Journal of Behavioral and Experimental Finance, Elsevier, vol. 42(C).
    31. Antonio A. Arechar & Simon Gächter & Lucas Molleman, 2018. "Conducting interactive experiments online," Experimental Economics, Springer;Economic Science Association, vol. 21(1), pages 99-131, March.
    32. Luz, Valentin & Schauer, Victor & Viehweger, Martin, 2024. "Beyond preferences: Beliefs in sustainable investing," Journal of Economic Behavior & Organization, Elsevier, vol. 220(C), pages 584-607.
    33. Matthew J C Crump & John V McDonnell & Todd M Gureckis, 2013. "Evaluating Amazon's Mechanical Turk as a Tool for Experimental Behavioral Research," PLOS ONE, Public Library of Science, vol. 8(3), pages 1-18, March.
    34. Bachler, Sebastian & Flecke, Sarah Lynn & Huber, Jürgen & Kirchler, Michael & Schwaiger, Rene, 2024. "Carbon Pricing, Carbon Dividends and Cooperation: Experimental Evidence," Journal of Economic Behavior & Organization, Elsevier, vol. 225(C), pages 37-50.
    35. Ozan Isler & Onurcan Yilmaz & A. John Maule, 2021. "Religion, parochialism and intuitive cooperation," Nature Human Behaviour, Nature, vol. 5(4), pages 512-521, April.
    36. Palan, Stefan & Schitter, Christian, 2018. "Prolific.ac—A subject pool for online experiments," Journal of Behavioral and Experimental Finance, Elsevier, vol. 17(C), pages 22-27.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gandullia, Luca & Lezzi, Emanuela & Parciasepe, Paolo, 2020. "Replication with MTurk of the experimental design by Gangadharan, Grossman, Jones & Leister (2018): Charitable giving across donor types," Journal of Economic Psychology, Elsevier, vol. 78(C).
    2. Palan, Stefan & Schitter, Christian, 2018. "Prolific.ac—A subject pool for online experiments," Journal of Behavioral and Experimental Finance, Elsevier, vol. 17(C), pages 22-27.
    3. Eriksen, Kristoffer W. & Fest, Sebastian & Kvaløy, Ola & Dijk, Oege, 2022. "Fair advice," Journal of Banking & Finance, Elsevier, vol. 143(C).
    4. Prissé, Benjamin & Jorrat, Diego, 2022. "Lab vs online experiments: No differences," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 100(C).
    5. Bhatt, Vipul & Smith, Angela M., 2025. "Overconfidence and performance: Evidence from a simple real-effort task," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 114(C).
    6. Brunner, Fabian & Gamm, Fabian & Mill, Wladislaw, 2023. "MyPortfolio: The IKEA effect in financial investment decisions," Journal of Banking & Finance, Elsevier, vol. 154(C).
    7. Hiroki Ozono & Daisuke Nakama, 2022. "Effects of experimental situation on group cooperation and individual performance: Comparing laboratory and online experiments," PLOS ONE, Public Library of Science, vol. 17(4), pages 1-17, April.
    8. Van Borm, Hannah & Burn, Ian & Baert, Stijn, 2021. "What Does a Job Candidate's Age Signal to Employers?," Labour Economics, Elsevier, vol. 71(C).
    9. Wladislaw Mill & John Morgan, 2022. "The cost of a divided America: an experimental study into destructive behavior," Experimental Economics, Springer;Economic Science Association, vol. 25(3), pages 974-1001, June.
    10. Guo, Yiting & Shachat, Jason & Walker, Matthew J. & Wei, Lijia, 2023. "On the generalizability of using mobile devices to conduct economic experiments," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 106(C).
    11. Antonio A. Arechar & Gordon T. Kraft-Todd & David G. Rand, 2017. "Turking overtime: how participant characteristics and behavior vary over time and day on Amazon Mechanical Turk," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 3(1), pages 1-11, July.
    12. Capraro, Valerio & Rodriguez-Lara, Ismael, 2025. "Moral preferences in ultimatum and impunity games," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 117(C).
    13. Milosav, Đorđe & Nistotskaya, Marina, 2024. "Unpacking the relevance of interpersonal trust in the blockchain era: Theory and experimental evidence," Journal of Behavioral and Experimental Finance, Elsevier, vol. 42(C).
    14. Hyndman, Kyle & Walker, Matthew J., 2022. "Fairness and risk in ultimatum bargaining," Games and Economic Behavior, Elsevier, vol. 132(C), pages 90-105.
    15. Burdea, Valeria & Woon, Jonathan, 2022. "Online belief elicitation methods," Journal of Economic Psychology, Elsevier, vol. 90(C).
    16. Prissé, Benjamin & Jorrat, Diego, 2021. "Lack of Control: An experiment," MPRA Paper 109918, University Library of Munich, Germany.
    17. Brañas-Garza, Pablo & Capraro, Valerio & Rascón-Ramírez, Ericka, 2018. "Gender differences in altruism on Mechanical Turk: Expectations and actual behaviour," Economics Letters, Elsevier, vol. 170(C), pages 19-23.
    18. Marcus Giamattei & Kyanoush Seyed Yahosseini & Simon Gächter & Lucas Molleman, 2020. "LIONESS Lab: a free web-based platform for conducting interactive experiments online," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 6(1), pages 95-111, June.
    19. Luz, Valentin & Schauer, Victor & Viehweger, Martin, 2024. "Beyond preferences: Beliefs in sustainable investing," Journal of Economic Behavior & Organization, Elsevier, vol. 220(C), pages 584-607.
    20. Valerio Capraro & Andrea Vanzo & Antonio Cabrales, 2022. "Playing with words: Do people exploit loaded language to affect others’ decisions for their own benefit?," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 17(1), pages 50-69, January.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:beexfi:v:48:y:2025:i:c:s2214635025000991. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/journal-of-behavioral-and-experimental-finance .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.