IDEAS home Printed from https://ideas.repec.org/a/eee/beexfi/v17y2018icp22-27.html
   My bibliography  Save this article

Prolific.ac—A subject pool for online experiments

Author

Listed:
  • Palan, Stefan
  • Schitter, Christian

Abstract

The number of online experiments conducted with subjects recruited via online platforms has grown considerably in the recent past. While one commercial crowdworking platform – Amazon’s Mechanical Turk – basically has established and since dominated this field, new alternatives offer services explicitly targeted at researchers. In this article, we present www.prolific.ac and lay out its suitability for recruiting subjects for social and economic science experiments. After briefly discussing key advantages and challenges of online experiments relative to lab experiments, we trace the platform’s historical development, present its features, and contrast them with requirements for different types of social and economic experiments.

Suggested Citation

  • Palan, Stefan & Schitter, Christian, 2018. "Prolific.ac—A subject pool for online experiments," Journal of Behavioral and Experimental Finance, Elsevier, vol. 17(C), pages 22-27.
  • Handle: RePEc:eee:beexfi:v:17:y:2018:i:c:p:22-27
    DOI: 10.1016/j.jbef.2017.12.004
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S2214635017300989
    Download Restriction: no

    References listed on IDEAS

    as
    1. Benndorf, Volker & Moellers, Claudia & Normann, Hans-Theo, 2017. "Experienced vs. inexperienced participants in the lab: Do they behave differently?," DICE Discussion Papers 251, University of Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
    2. John Horton & David Rand & Richard Zeckhauser, 2011. "The online laboratory: conducting experiments in a real labor market," Experimental Economics, Springer;Economic Science Association, vol. 14(3), pages 399-425, September.
    3. Landers, Richard N. & Behrend, Tara S., 2015. "An Inconvenient Truth: Arbitrary Distinctions Between Organizational, Mechanical Turk, and Other Convenience Samples," Industrial and Organizational Psychology, Cambridge University Press, vol. 8(2), pages 142-164, June.
    4. Antonio A. Arechar & Simon Gächter & Lucas Molleman, 2018. "Conducting interactive experiments online," Experimental Economics, Springer;Economic Science Association, vol. 21(1), pages 99-131, March.
    5. Urs Fischbacher & Franziska Föllmi-Heusi, 2013. "Lies In Disguise—An Experimental Study On Cheating," Journal of the European Economic Association, European Economic Association, vol. 11(3), pages 525-547, June.
    6. Johannes Abeler & Armin Falk & Lorenz Goette & David Huffman, 2011. "Reference Points and Effort Provision," American Economic Review, American Economic Association, vol. 101(2), pages 470-492, April.
    7. Gabriele Paolacci & Jesse Chandler & Panagiotis G. Ipeirotis, 2010. "Running experiments on Amazon Mechanical Turk," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 5(5), pages 411-419, August.
    8. Ofra Amir & David G Rand & Ya'akov Kobi Gal, 2012. "Economic Games on the Internet: The Effect of $1 Stakes," PLOS ONE, Public Library of Science, vol. 7(2), pages 1-4, February.
    9. Volker Benndorf & Claudia Moellers & Hans-Theo Normann, 2017. "Experienced vs. inexperienced participants in the lab: do they behave differently?," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 3(1), pages 12-25, July.
    10. Ben Greiner, 2015. "Subject pool recruitment procedures: organizing experiments with ORSEE," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 1(1), pages 114-125, July.
    11. Marreiros, Helia & Tonin, Mirco & Vlassopoulos, Michael & Schraefel, M.C., 2017. "“Now that you mention it”: A survey experiment on information, inattention and online privacy," Journal of Economic Behavior & Organization, Elsevier, vol. 140(C), pages 1-17.
    12. Neil Stewart & Christoph Ungemach & Adam J. L. Harris & Daniel M. Bartels & Ben R. Newell & Gabriele Paolacci & Jesse Chandler, 2015. "The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 10(5), pages 479-491, September.
    13. Siddharth Suri & Duncan J Watts, 2011. "Cooperation and Contagion in Web-Based, Networked Public Goods Experiments," PLOS ONE, Public Library of Science, vol. 6(3), pages 1-18, March.
    14. Cooper, David J., 2014. "A Note on Deception in Economic Experiments," Journal of Wine Economics, Cambridge University Press, vol. 9(02), pages 111-114, August.
    15. Matthew J C Crump & John V McDonnell & Todd M Gureckis, 2013. "Evaluating Amazon's Mechanical Turk as a Tool for Experimental Behavioral Research," PLOS ONE, Public Library of Science, vol. 8(3), pages 1-18, March.
    16. Elizabeth A Necka & Stephanie Cacioppo & Greg J Norman & John T Cacioppo, 2016. "Measuring the Prevalence of Problematic Respondent Behaviors among MTurk, Campus, and Community Participants," PLOS ONE, Public Library of Science, vol. 11(6), pages 1-19, June.
    17. Chen, Daniel L. & Schonger, Martin & Wickens, Chris, 2016. "oTree—An open-source platform for laboratory, online, and field experiments," Journal of Behavioral and Experimental Finance, Elsevier, vol. 9(C), pages 88-97.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marco Catola & Simone D'Alessandro & Pietro Guarnieri & Veronica Pizziol, 2020. "Multilevel Public Goods Game: an Online Experiment," Discussion Papers 2020/263, Dipartimento di Economia e Management (DEM), University of Pisa, Pisa, Italy.
    2. Barrafrem, Kinga & Västfjäll, Daniel & Tinghög, Gustav, 2020. "Financial well-being, COVID-19, and the financial better-than-average-effect," Journal of Behavioral and Experimental Finance, Elsevier, vol. 28(C).
    3. Huber, Christoph & Huber, Jürgen, 2020. "Bad bankers no more? Truth-telling and (dis)honesty in the finance industry," Journal of Economic Behavior & Organization, Elsevier, vol. 180(C), pages 472-493.
    4. Donavon Johnson & Alexander Kroll, 2021. "What makes us tolerant of administrative burden? Race, representation, and identity," Journal of Behavioral Public Administration, Center for Experimental and Behavioral Public Administration, vol. 4(1).
    5. Dickinson, David L., 2020. "Deliberation Enhances the Confirmation Bias: An Examination of Politics and Religion," IZA Discussion Papers 13241, Institute of Labor Economics (IZA).
    6. Silvia Saccardo & Marta Serra-Garcia, 2020. "Cognitive Flexibility or Moral Commitment? Evidence of Anticipated Belief Distortion," CESifo Working Paper Series 8529, CESifo.
    7. Lilleholt, Lau & Schild, Christoph & Zettler, Ingo, 2020. "Not all computerized cheating tasks are equal: A comparison of computerized and non-computerized versions of a cheating task," Journal of Economic Psychology, Elsevier, vol. 78(C).
    8. Chan, Nathan W. & Wolk, Leonard, 2020. "Cost-effective giving with multiple public goods," Journal of Economic Behavior & Organization, Elsevier, vol. 173(C), pages 130-145.
    9. Chapkovski, Philipp & Zihlmann, Christian, 2019. "Introducing otree_tools: A powerful package to provide process data for attention, multitasking behavior and effort through tracking focus," Journal of Behavioral and Experimental Finance, Elsevier, vol. 23(C), pages 75-83.
    10. Collewaert, Veroniek & Vanacker, Tom & Anseel, Frederik & Bourgois, Dries, 2021. "The sandwich game: Founder-CEOs and forecasting as impression management," Journal of Business Venturing, Elsevier, vol. 36(1).
    11. Blanco, Fernando & Matute, Helena, 2020. "Diseases that resolve spontaneously can increase the belief that ineffective treatments work," Social Science & Medicine, Elsevier, vol. 255(C).
    12. Kobayashi, Yoshiharu & Heinrich, Tobias & Bryant, Kristin A., 2021. "Public support for development aid during the COVID-19 pandemic," World Development, Elsevier, vol. 138(C).
    13. Engel, Yuval & Ramesh, Anusha & Steiner, Nick, 2020. "Powered by compassion: The effect of loving-kindness meditation on entrepreneurs' sustainable decision-making," Journal of Business Venturing, Elsevier, vol. 35(6).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Benndorf, Volker & Rau, Holger A. & Sölch, Christian, 2019. "Minimizing learning in repeated real-effort tasks," Journal of Behavioral and Experimental Finance, Elsevier, vol. 22(C), pages 239-248.
    2. Capraro, Valerio & Schulz, Jonathan & Rand, David G., 2019. "Time pressure and honesty in a deception game," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 79(C), pages 93-99.
    3. Gandullia, Luca & Lezzi, Emanuela & Parciasepe, Paolo, 2020. "Replication with MTurk of the experimental design by Gangadharan, Grossman, Jones & Leister (2018): Charitable giving across donor types," Journal of Economic Psychology, Elsevier, vol. 78(C).
    4. Dato, Simon & Feess, Eberhard & Nieken, Petra, 2019. "Lying and reciprocity," Games and Economic Behavior, Elsevier, vol. 118(C), pages 193-218.
    5. Irene Maria Buso & Daniela Di Cagno & Sofia De Caprariis & Lorenzo Ferrari & Vittorio Larocca & Luisa Lorè & Francesca Marazzi & Luca Panaccione & Lorenzo Spadoni, 2020. "Lab-like Findings of Non-Lab Experiments: a Methodological Proposal and Validation," Working Papers CESARE 3/2020, Dipartimento di Economia e Finanza, LUISS Guido Carli.
    6. Irene Maria Buso & Daniela Di Cagno & Sofia De Caprariis & Lorenzo Ferrari & Vittorio Larocca & Francesca Marazzi & Luca Panaccione & Lorenzo Spadoni, 2020. "The Show Must Go On: How to Elicit Lablike Data on the Effects of COVID-19 Lockdown on Fairness and Cooperation," Working Papers CESARE 2/2020, Dipartimento di Economia e Finanza, LUISS Guido Carli.
    7. Rebecca R Carter & Analisa DiFeo & Kath Bogie & Guo-Qiang Zhang & Jiayang Sun, 2014. "Crowdsourcing Awareness: Exploration of the Ovarian Cancer Knowledge Gap through Amazon Mechanical Turk," PLOS ONE, Public Library of Science, vol. 9(1), pages 1-10, January.
    8. Frederiks, Arjan J. & Englis, Basil G. & Ehrenhard, Michel L. & Groen, Aard J., 2019. "Entrepreneurial cognition and the quality of new venture ideas: An experimental approach to comparing future-oriented cognitive processes," Journal of Business Venturing, Elsevier, vol. 34(2), pages 327-347.
    9. Heinicke, Franziska & Rosenkranz, Stephanie & Weitzel, Utz, 2019. "The effect of pledges on the distribution of lying behavior: An online experiment," Journal of Economic Psychology, Elsevier, vol. 73(C), pages 136-151.
    10. Brañas-Garza, Pablo & Capraro, Valerio & Rascón-Ramírez, Ericka, 2018. "Gender differences in altruism on Mechanical Turk: Expectations and actual behaviour," Economics Letters, Elsevier, vol. 170(C), pages 19-23.
    11. Ellen Garbarino & Robert Slonim & Marie Claire Villeval, 2018. "A method to estimate mean lying rates and their full distribution," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 4(2), pages 136-150, December.
    12. Hindsley, Paul & McEvoy, David M. & Morgan, O. Ashton, 2020. "Consumer Demand for Ethical Products and the Role of Cultural Worldviews: The Case of Direct-Trade Coffee," Ecological Economics, Elsevier, vol. 177(C).
    13. Jillian J Jordan & David G Rand & Samuel Arbesman & James H Fowler & Nicholas A Christakis, 2013. "Contagion of Cooperation in Static and Fluid Social Networks," PLOS ONE, Public Library of Science, vol. 8(6), pages 1-10, June.
    14. Joseph A. Johnson & Jochen Theis & Adam Vitalis & Donald Young, 2020. "The Influence of Firms' Emissions Management Strategy Disclosures on Investors' Valuation Judgments†," Contemporary Accounting Research, John Wiley & Sons, vol. 37(2), pages 642-664, June.
    15. Valerio Capraro, 2018. "Gender differences in lying in sender-receiver games: A meta-analysis," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 13(4), pages 345-355, July.
    16. Antonio A. Arechar & Simon Gächter & Lucas Molleman, 2018. "Conducting interactive experiments online," Experimental Economics, Springer;Economic Science Association, vol. 21(1), pages 99-131, March.
    17. Garbarino, Ellen & Slonim, Robert & Villeval, Marie Claire, 2019. "Loss aversion and lying behavior," Journal of Economic Behavior & Organization, Elsevier, vol. 158(C), pages 379-393.
    18. Ofra Amir & David G Rand & Ya'akov Kobi Gal, 2012. "Economic Games on the Internet: The Effect of $1 Stakes," PLOS ONE, Public Library of Science, vol. 7(2), pages 1-4, February.
    19. Alempaki, Despoina & Starmer, Chris & Tufano, Fabio, 2019. "On the priming of risk preferences: The role of fear and general affect," Journal of Economic Psychology, Elsevier, vol. 75(PA).
    20. Hardin, Ashley E. & Bauman, Christopher W. & Mayer, David M., 2020. "Show me the … family: How photos of meaningful relationships reduce unethical behavior at work," Organizational Behavior and Human Decision Processes, Elsevier, vol. 161(C), pages 93-108.

    More about this item

    Keywords

    Prolific; Online experiment; Subject pool;
    All these keywords.

    JEL classification:

    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
    • B41 - Schools of Economic Thought and Methodology - - Economic Methodology - - - Economic Methodology

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:beexfi:v:17:y:2018:i:c:p:22-27. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Haili He). General contact details of provider: https://www.journals.elsevier.com/journal-of-behavioral-and-experimental-finance .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.