Advanced Search
MyIDEAS: Login

Running experiments on Amazon Mechanical Turk

Contents:

Author Info

  • Gabriele Paolacci
  • Jesse Chandler
  • Panagiotis G. Ipeirotis
Registered author(s):

    Abstract

    Although Mechanical Turk has recently become popular among social scientists as a source of experimental data, doubts may linger about the quality of data provided by subjects recruited from online labor markets. We address these potential concerns by presenting new demographic data about the Mechanical Turk subject population, reviewing the strengths of Mechanical Turk relative to other online and offline methods of recruiting subjects, and comparing the magnitude of effects obtained using Mechanical Turk and traditional subject pools. We further discuss some additional benefits such as the possibility of longitudinal, cross cultural and prescreening designs, and offer some advice on how to best manage a common subject pool.

    Download Info

    If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
    File URL: http://journal.sjdm.org/10/10630a/jdm10630a.pdf
    Download Restriction: no

    File URL: http://journal.sjdm.org/10/10630a/jdm10630a.html
    Download Restriction: no

    Bibliographic Info

    Article provided by Society for Judgment and Decision Making in its journal Judgment and Decision Making.

    Volume (Year): 5 (2010)
    Issue (Month): 5 (August)
    Pages: 411-419

    as in new window
    Handle: RePEc:jdm:journl:v:5:y:2010:i:5:p:411-419

    Contact details of provider:

    Related research

    Keywords: experimentation; online research;

    References

    No references listed on IDEAS
    You can help add them by filling out this form.

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as in new window

    Cited by:
    1. Yamada, Katsunori & Sato, Masayuki, 2013. "Another avenue for anatomy of income comparisons: Evidence from hypothetical choice experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 89(C), pages 35-57.
    2. Chandler, Dana & Kapelner, Adam, 2013. "Breaking monotony with meaning: Motivation in crowdsourcing markets," Journal of Economic Behavior & Organization, Elsevier, vol. 90(C), pages 123-133.
    3. Aguinis, Herman & Lawal, Sola O., 2012. "Conducting field experiments using eLancing's natural environment," Journal of Business Venturing, Elsevier, vol. 27(4), pages 493-505.
    4. Dreber Almenberg, Anna & Gerdes, Christer & Gränsmark, Patrik, 2010. "Beauty Queens and Battling Knights: Risk Taking and Attractiveness in Chess," IZA Discussion Papers 5314, Institute for the Study of Labor (IZA).
    5. Wieland, Alice & Sarin, Rakesh, 2012. "Domain specificity of sex differences in competition," Journal of Economic Behavior & Organization, Elsevier, vol. 83(1), pages 151-157.
    6. Sharma, Eesha & Mazar, Nina & Alter, Adam L. & Ariely, Dan, 2014. "Financial deprivation selectively shifts moral standards and compromises moral decisions," Organizational Behavior and Human Decision Processes, Elsevier, vol. 123(2), pages 90-100.
    7. Charness, Gary & Gneezy, Uri & Kuhn, Michael A., 2013. "Experimental methods: Extra-laboratory experiments-extending the reach of experimental economics," Journal of Economic Behavior & Organization, Elsevier, vol. 91(C), pages 93-100.
    8. Gupta, Vishal K. & Goktan, A. Banu & Gunay, Gonca, 2014. "Gender differences in evaluation of new business opportunity: A stereotype threat perspective," Journal of Business Venturing, Elsevier, vol. 29(2), pages 273-288.
    9. Polman, Evan, 2012. "Self–other decision making and loss aversion," Organizational Behavior and Human Decision Processes, Elsevier, vol. 119(2), pages 141-150.
    10. Michael Boyer O'leary & Anca Metiu & Jeanne M. Wilson, 2011. "Beyond Being There: The Symbolic Role of Communication and Identification in the Emergence of Perceived Proximity in Geographically Dispersed Work," Post-Print hal-00661000, HAL.
    11. David Cooper & David Johnson, 2013. "Ambiguity in Performance Pay: An Online Experiment," Working Papers 2013-27, Department of Economics, University of Calgary, revised 14 Nov 2013.
    12. Manapat, Michael L. & Nowak, Martin A. & Rand, David G., 2013. "Information, irrationality, and the evolution of trust," Journal of Economic Behavior & Organization, Elsevier, vol. 90(S), pages S57-S75.

    Lists

    This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

    Statistics

    Access and download statistics

    Corrections

    When requesting a correction, please mention this item's handle: RePEc:jdm:journl:v:5:y:2010:i:5:p:411-419. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Jonathan Baron).

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If references are entirely missing, you can add them using this form.

    If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.