IDEAS home Printed from https://ideas.repec.org/p/mpr/mprres/f97b669c7b3e4c2ab95c9f8051d18af6.html

The Average Laboratory Samples a Population of 7,300 Amazon Mechanical Turk Workers

Author

Listed:
  • Neil Stewart
  • Christoph Ungemach
  • Adam J. L. Harris
  • Daniel M. Bartels
  • Ben R. Newell
  • Gabriele Paolacci
  • Jesse Chandler

Abstract

Using capture-recapture analysis (a tool used by ecologists to estimate the size of hard to observe populations) we estimate the effective size of the active Amazon Mechanical Turk (MTurk) population that a typical laboratory can access to be about 7,300 workers.

Suggested Citation

  • Neil Stewart & Christoph Ungemach & Adam J. L. Harris & Daniel M. Bartels & Ben R. Newell & Gabriele Paolacci & Jesse Chandler, "undated". "The Average Laboratory Samples a Population of 7,300 Amazon Mechanical Turk Workers," Mathematica Policy Research Reports f97b669c7b3e4c2ab95c9f805, Mathematica Policy Research.
    • Handle: RePEc:mpr:mprres:f97b669c7b3e4c2ab95c9f8051d18af6
    as

    Download full text from publisher

    File URL: http://journal.sjdm.org/14/14725/jdm14725.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. David G. Rand & Alexander Peysakhovich & Gordon T. Kraft-Todd & George E. Newman & Owen Wurzbacher & Martin A. Nowak & Joshua D. Greene, 2014. "Social heuristics shape intuitive cooperation," Nature Communications, Nature, vol. 5(1), pages 1-12, May.
    2. Gabriele Paolacci & Jesse Chandler & Panagiotis G. Ipeirotis, 2010. "Running experiments on Amazon Mechanical Turk," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 5(5), pages 411-419, August.
    3. Baillargeon, Sophie & Rivest, Louis-Paul, 2007. "Rcapture: Loglinear Models for Capture-Recapture in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 19(i05).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Capraro, Valerio & Schulz, Jonathan & Rand, David G., 2019. "Time pressure and honesty in a deception game," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 79(C), pages 93-99.
    2. Stevenson, Regan M. & Josefy, Matthew, 2019. "Knocking at the gate: The path to publication for entrepreneurship experiments through the lens of gatekeeping theory," Journal of Business Venturing, Elsevier, vol. 34(2), pages 242-260.
    3. Sergio Alessandro Castagnetti & Sebastiano Massaro & Eugenio Proto, 2021. "The Influence of Anger on Strategic Cooperative Interactions," Working Papers 2021_05, Business School - Economics, University of Glasgow.
    4. Logan S. Casey & Jesse Chandler & Adam Seth Levine & Andrew Proctor & Dara Z. Strolovitch, 2017. "Intertemporal Differences Among MTurk Workers: Time-Based Sample Variations and Implications for Online Data Collection," SAGE Open, , vol. 7(2), pages 21582440177, June.
    5. Christ, Margaret H. & Vance, Thomas W., 2018. "Cascading controls: The effects of managers’ incentives on subordinate effort to help or harm," Accounting, Organizations and Society, Elsevier, vol. 65(C), pages 20-32.
    6. Antonio A. Arechar & Simon Gächter & Lucas Molleman, 2018. "Conducting interactive experiments online," Experimental Economics, Springer;Economic Science Association, vol. 21(1), pages 99-131, March.
    7. Brandi S. Morris & Polymeros Chrysochou & Jacob Dalgaard Christensen & Jacob L. Orquin & Jorge Barraza & Paul J. Zak & Panagiotis Mitkidis, 2019. "Stories vs. facts: triggering emotion and action-taking on climate change," Climatic Change, Springer, vol. 154(1), pages 19-36, May.
    8. Roman Lukyanenko & Andrea Wiggins & Holly K. Rosser, 0. "Citizen Science: An Information Quality Research Frontier," Information Systems Frontiers, Springer, vol. 0, pages 1-23.
    9. Austin M Strange & Ryan D Enos & Mark Hill & Amy Lakeman, 2019. "Online volunteer laboratories for human subjects research," PLOS ONE, Public Library of Science, vol. 14(8), pages 1-13, August.
    10. Tom Meyvis & Stijn M J Van Osselaer & Dahl DarrenEditor & Eileen FischerEditor & Gita JoharEditor & Vicki MorwitzEditor, 2018. "Increasing the Power of Your Study by Increasing the Effect Size," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 44(5), pages 1157-1173.
    11. Clément Le Ludec & Paola Tubaro & Antonio A. Casilli, 2019. "How many people microwork in France? Estimating the size of a new labor force," Working Papers hal-02012731, HAL.
    12. Fiona Charnley & Fabienne Knecht & Helge Muenkel & Diana Pletosu & Victoria Rickard & Chiara Sambonet & Martina Schneider & Chunli Zhang, 2022. "Can Digital Technologies Increase Consumer Acceptance of Circular Business Models? The Case of Second Hand Fashion," Sustainability, MDPI, vol. 14(8), pages 1-21, April.
    13. Cloos, Janis & Greiff, Matthias & Rusch, Hannes, 2019. "Geographical Concentration and Editorial Favoritism within the Field of Laboratory Experimental Economics," Research Memorandum 029, Maastricht University, Graduate School of Business and Economics (GSBE).
    14. Palan, Stefan & Schitter, Christian, 2018. "Prolific.ac—A subject pool for online experiments," Journal of Behavioral and Experimental Finance, Elsevier, vol. 17(C), pages 22-27.
    15. Surti, Chirag & Celani, Anthony & Gajpal, Yuvraj, 2020. "The newsvendor problem: The role of prospect theory and feedback," European Journal of Operational Research, Elsevier, vol. 287(1), pages 251-261.
    16. Anthony M. Evans & Joachim I. Krueger, 2017. "Ambiguity and expectation-neglect in dilemmas of interpersonal trust," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 12(6), pages 584-595, November.
    17. Cloos, Janis & Greiff, Matthias & Rusch, Hannes, 2020. "Geographical Concentration and Editorial Favoritism within the Field of Laboratory Experimental Economics (RM/19/029-revised-)," Research Memorandum 014, Maastricht University, Graduate School of Business and Economics (GSBE).
    18. Kevin C Elliott & Aaron M McCright & Summer Allen & Thomas Dietz, 2017. "Values in environmental research: Citizens’ views of scientists who acknowledge values," PLOS ONE, Public Library of Science, vol. 12(10), pages 1-18, October.
    19. David Ronayne & Daniel Sgroi, 2018. "On the motivations for the dual-use of electronic and traditional cigarettes," Applied Economics Letters, Taylor & Francis Journals, vol. 25(12), pages 830-834, July.
    20. Clément Le Ludec & Paola Tubaro & Antonio A. Casilli, 2019. "Combien de personnes micro-travaillent en France ? Estimer l'ampleur d'une nouvelle forme de travail," Working Papers hal-02021525, HAL.
    21. Keela S. Thomson & Daniel M. Oppenheimer, 2016. "Investigating an alternate form of the cognitive reflection test," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 11(1), pages 99-113, January.
    22. Roman Lukyanenko & Andrea Wiggins & Holly K. Rosser, 2020. "Citizen Science: An Information Quality Research Frontier," Information Systems Frontiers, Springer, vol. 22(4), pages 961-983, August.
    23. Xiaochuan Song & Graham H. Lowman & Peter Harms, 2020. "Justice for the Crowd: Organizational Justice and Turnover in Crowd-Based Labor," Administrative Sciences, MDPI, vol. 10(4), pages 1-37, November.
    24. Brian P. Meier & Amanda J. Dillard & Eric Osorio & Courtney M. Lappas, 2019. "A Behavioral Confirmation and Reduction of the Natural versus Synthetic Drug Bias," Medical Decision Making, , vol. 39(4), pages 360-370, May.
    25. Summer Allen & Aaron M. McCright & Thomas Dietz, 2017. "A Social Movement Identity Instrument for Integrating Survey Methods Into Social Movements Research," SAGE Open, , vol. 7(2), pages 21582440177, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Neil Stewart & Christoph Ungemach & Adam J. L. Harris & Daniel M. Bartels & Ben R. Newell & Gabriele Paolacci & Jesse Chandler, 2015. "The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 10(5), pages 479-491, September.
    2. Wladislaw Mill & Cornelius Schneider, 2023. "The Bright Side of Tax Evasion," CESifo Working Paper Series 10615, CESifo.
    3. Johannes Lohse & Timo Goeschl & Johannes H. Diederich, 2017. "Giving is a Question of Time: Response Times and Contributions to an Environmental Public Good," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 67(3), pages 455-477, July.
    4. Brunner, Fabian & Gamm, Fabian & Mill, Wladislaw, 2023. "MyPortfolio: The IKEA effect in financial investment decisions," Journal of Banking & Finance, Elsevier, vol. 154(C).
    5. Antonio A. Arechar & Simon Gächter & Lucas Molleman, 2018. "Conducting interactive experiments online," Experimental Economics, Springer;Economic Science Association, vol. 21(1), pages 99-131, March.
    6. Wladislaw Mill & John Morgan, 2022. "The cost of a divided America: an experimental study into destructive behavior," Experimental Economics, Springer;Economic Science Association, vol. 25(3), pages 974-1001, June.
    7. David G Rand & Gordon Kraft-Todd & June Gruber, 2015. "The Collective Benefits of Feeling Good and Letting Go: Positive Emotion and (dis)Inhibition Interact to Predict Cooperative Behavior," PLOS ONE, Public Library of Science, vol. 10(1), pages 1-12, January.
    8. Wladislaw Mill & Jonathan Staebler, 2023. "Spite in Litigation," CRC TR 224 Discussion Paper Series crctr224_2023_401, University of Bonn and University of Mannheim, Germany.
    9. Antonio A. Arechar & Gordon T. Kraft-Todd & David G. Rand, 2017. "Turking overtime: how participant characteristics and behavior vary over time and day on Amazon Mechanical Turk," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 3(1), pages 1-11, July.
    10. Capraro, Valerio & Schulz, Jonathan & Rand, David G., 2019. "Time pressure and honesty in a deception game," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 79(C), pages 93-99.
    11. Capraro, Valerio & Kuilder, Jotte, 2016. "To know or not to know? Looking at payoffs signals selfish behavior, but it does not actually mean so," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 65(C), pages 79-84.
    12. Jeremy Cone & David G Rand, 2014. "Time Pressure Increases Cooperation in Competitively Framed Social Dilemmas," PLOS ONE, Public Library of Science, vol. 9(12), pages 1-13, December.
    13. Hoeft, Leonard & Kurschilgen, Michael & Mill, Wladislaw, 2025. "Norms as obligations," International Review of Law and Economics, Elsevier, vol. 81(C).
    14. Yamada, Katsunori & Sato, Masayuki, 2013. "Another avenue for anatomy of income comparisons: Evidence from hypothetical choice experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 89(C), pages 35-57.
    15. Kenneth D. Nguyen & Heather Rosoff & Richard S. John, 2017. "Valuing Equal Protection in Aviation Security Screening," Risk Analysis, John Wiley & Sons, vol. 37(12), pages 2405-2419, December.
    16. Varun Dutt & Cleotilde Gonzalez, 2013. "Enabling Eco-Friendly Choices by Relying on the Proportional-Thinking Heuristic," Sustainability, MDPI, vol. 5(1), pages 1-15, January.
    17. Sweldens, Steven & Puntoni, Stefano & Paolacci, Gabriele & Vissers, Maarten, 2014. "The bias in the bias: Comparative optimism as a function of event social undesirability," Organizational Behavior and Human Decision Processes, Elsevier, vol. 124(2), pages 229-244.
    18. Christian Hilbe & Moshe Hoffman & Martin A. Nowak, 2015. "Cooperate without Looking in a Non-Repeated Game," Games, MDPI, vol. 6(4), pages 1-15, September.
    19. S. Venus Jin & Aziz Muqaddam, 2019. "Product placement 2.0: “Do Brands Need Influencers, or Do Influencers Need Brands?”," Journal of Brand Management, Palgrave Macmillan, vol. 26(5), pages 522-537, September.
    20. Jacob W. Crandall & Mayada Oudah & Tennom & Fatimah Ishowo-Oloko & Sherief Abdallah & Jean-François Bonnefon & Manuel Cebrian & Azim Shariff & Michael A. Goodrich & Iyad Rahwan, 2018. "Cooperating with machines," Nature Communications, Nature, vol. 9(1), pages 1-12, December.
      • Abdallah, Sherief & Bonnefon, Jean-François & Cebrian, Manuel & Crandall, Jacob W. & Ishowo-Oloko, Fatimah & Oudah, Mayada & Rahwan, Iyad & Shariff, Azim & Tennom,, 2017. "Cooperating with Machines," TSE Working Papers 17-806, Toulouse School of Economics (TSE).
      • Abdallah, Sherief & Bonnefon, Jean-François & Cebrian, Manuel & Crandall, Jacob W. & Ishowo-Oloko, Fatimah & Oudah, Mayada & Rahwan, Iyad & Shariff, Azim & Tennom,, 2017. "Cooperating with Machines," IAST Working Papers 17-68, Institute for Advanced Study in Toulouse (IAST).

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:mpr:mprres:f97b669c7b3e4c2ab95c9f8051d18af6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joanne Pfleiderer or Cindy George The email address of this maintainer does not seem to be valid anymore. Please ask Cindy George to update the entry or send us the correct address (email available below). General contact details of provider: https://edirc.repec.org/data/mathius.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.