Advanced Search
MyIDEAS: Login to save this article or follow this journal

Running experiments on Amazon Mechanical Turk

Contents:

Author Info

  • Gabriele Paolacci
  • Jesse Chandler
  • Panagiotis G. Ipeirotis
Registered author(s):

    Abstract

    Although Mechanical Turk has recently become popular among social scientists as a source of experimental data, doubts may linger about the quality of data provided by subjects recruited from online labor markets. We address these potential concerns by presenting new demographic data about the Mechanical Turk subject population, reviewing the strengths of Mechanical Turk relative to other online and offline methods of recruiting subjects, and comparing the magnitude of effects obtained using Mechanical Turk and traditional subject pools. We further discuss some additional benefits such as the possibility of longitudinal, cross cultural and prescreening designs, and offer some advice on how to best manage a common subject pool.

    Download Info

    If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
    File URL: http://journal.sjdm.org/10/10630a/jdm10630a.pdf
    Download Restriction: no

    File URL: http://journal.sjdm.org/10/10630a/jdm10630a.html
    Download Restriction: no

    Bibliographic Info

    Article provided by Society for Judgment and Decision Making in its journal Judgment and Decision Making.

    Volume (Year): 5 (2010)
    Issue (Month): 5 (August)
    Pages: 411-419

    as in new window
    Handle: RePEc:jdm:journl:v:5:y:2010:i:5:p:411-419

    Contact details of provider:

    Related research

    Keywords: experimentation; online research;

    References

    No references listed on IDEAS
    You can help add them by filling out this form.

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as in new window

    Cited by:
    1. repec:clg:wpaper:2013-27 is not listed on IDEAS
    2. Wieland, Alice & Sarin, Rakesh, 2012. "Domain specificity of sex differences in competition," Journal of Economic Behavior & Organization, Elsevier, vol. 83(1), pages 151-157.
    3. Dreber, Anna & Gerdes, Christer & Gränsmark, Patrik, 2013. "Beauty queens and battling knights: Risk taking and attractiveness in chess," Journal of Economic Behavior & Organization, Elsevier, vol. 90(C), pages 1-18.
    4. Polman, Evan, 2012. "Self–other decision making and loss aversion," Organizational Behavior and Human Decision Processes, Elsevier, vol. 119(2), pages 141-150.
    5. Boyer O’Leary, Michael & Wilson, Jeanne M. & Metiu, Anca, 2011. "Beyond Being There: The Symbolic Role of Communication and Identification in the Emergence of Perceived Proximity in Geographically Dispersed Work," ESSEC Working Papers WP1112, ESSEC Research Center, ESSEC Business School.
    6. Yamada, Katsunori & Sato, Masayuki, 2013. "Another avenue for anatomy of income comparisons: Evidence from hypothetical choice experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 89(C), pages 35-57.
    7. Charness, Gary & Gneezy, Uri & Kuhn, Michael A., 2013. "Experimental methods: Extra-laboratory experiments-extending the reach of experimental economics," Journal of Economic Behavior & Organization, Elsevier, vol. 91(C), pages 93-100.
    8. Yury Shevchenko & Bettina von Helversen & Benjamin Scheibehenne, 2014. "Change and status quo in decisions with defaults: The effect of incidental emotions depends on the type of default," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 9(3), pages 287-296, May.
    9. Katherine B. Coffman & Lucas C. Coffman & Keith M. Marzilli Ericson, 2013. "The Size of the LGBT Population and the Magnitude of Anti-Gay Sentiment are Substantially Underestimated," NBER Working Papers 19508, National Bureau of Economic Research, Inc.
    10. Manapat, Michael L. & Nowak, Martin A. & Rand, David G., 2013. "Information, irrationality, and the evolution of trust," Journal of Economic Behavior & Organization, Elsevier, vol. 90(S), pages S57-S75.
    11. Aguinis, Herman & Lawal, Sola O., 2012. "Conducting field experiments using eLancing's natural environment," Journal of Business Venturing, Elsevier, vol. 27(4), pages 493-505.
    12. Sharma, Eesha & Mazar, Nina & Alter, Adam L. & Ariely, Dan, 2014. "Financial deprivation selectively shifts moral standards and compromises moral decisions," Organizational Behavior and Human Decision Processes, Elsevier, vol. 123(2), pages 90-100.
    13. Chandler, Dana & Kapelner, Adam, 2013. "Breaking monotony with meaning: Motivation in crowdsourcing markets," Journal of Economic Behavior & Organization, Elsevier, vol. 90(C), pages 123-133.
    14. Gupta, Vishal K. & Goktan, A. Banu & Gunay, Gonca, 2014. "Gender differences in evaluation of new business opportunity: A stereotype threat perspective," Journal of Business Venturing, Elsevier, vol. 29(2), pages 273-288.
    15. Armenak Antinyan, 2014. "Loss and Other-Regarding Preferences: Evidence From Dictator Game," Working Papers 3, Department of Management, Università Ca' Foscari Venezia.

    Lists

    This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

    Statistics

    Access and download statistics

    Corrections

    When requesting a correction, please mention this item's handle: RePEc:jdm:journl:v:5:y:2010:i:5:p:411-419. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Jonathan Baron).

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If references are entirely missing, you can add them using this form.

    If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.