MTurk ‘Unscrubbed’: Exploring the good, the ‘Super’, and the unreliable on Amazon’s Mechanical Turk
Author
Abstract
Suggested Citation
Download full text from publisher
References listed on IDEAS
- John Horton & David Rand & Richard Zeckhauser, 2011.
"The online laboratory: conducting experiments in a real labor market,"
Experimental Economics, Springer;Economic Science Association, vol. 14(3), pages 399-425, September.
- John J. Horton & David G. Rand & Richard J. Zeckhauser, 2010. "The Online Laboratory: Conducting Experiments in a Real Labor Market," NBER Working Papers 15961, National Bureau of Economic Research, Inc.
- Horton, John Joseph & Rand, David Gertler & Zeckhauser, Richard Jay, 2010. "The Online Laboratory: Conducting Experiments in a Real Labor Market," Scholarly Articles 4448876, Harvard Kennedy School of Government.
- Horton, John J. & Rand, David G. & Zeckhauser, Richard, 2010. "The Online Laboratory: Conducting Experiments in a Real Labor Market," Working Paper Series rwp10-017, Harvard University, John F. Kennedy School of Government.
- van Rooij, Maarten & Lusardi, Annamaria & Alessie, Rob, 2011.
"Financial literacy and stock market participation,"
Journal of Financial Economics, Elsevier, vol. 101(2), pages 449-472, August.
- M.C.J. van Rooij & A. Lusardi & R. Alessie, 2007. "Financial Literacy and Stock Market Participation," Working Papers 07-23, Utrecht School of Economics.
- van Rooij, Maarten & Lusardi, Annamaria & Alessie, Rob J. M., 2007. "Financial literacy and stock market participation," CFS Working Paper Series 2007/27, Center for Financial Studies (CFS).
- Maarten van Rooij & Annamaria Lusardi & Rob Alessie, 2007. "Financial Literacy and Stock Market Participation," NBER Working Papers 13565, National Bureau of Economic Research, Inc.
- Maarten van Rooij & Annamaria Lusardi & Rob Alessie, 2007. "Financial Literacy and Stock Market Participation," CeRP Working Papers 66, Center for Research on Pensions and Welfare Policies, Turin (Italy).
- Maarten vanRooij & Annamaria Lusardi & Rob Alessie, 2007. "Financial Literacy and Stock Market Participation," Working Papers wp162, University of Michigan, Michigan Retirement Research Center.
- Todd L. Cherry & Peter Frykblom & Jason F. Shogren, 2002.
"Hardnose the Dictator,"
American Economic Review, American Economic Association, vol. 92(4), pages 1218-1221, September.
- Todd L. Cherry & Peter Frykblom & Jason F. Shogren, 2002. "Hardnose the Dictator," Working Papers 02-06, Department of Economics, Appalachian State University.
- Lusardi, Annamaria & Mitchell, Olivia S., 2007.
"Baby Boomer retirement security: The roles of planning, financial literacy, and housing wealth,"
Journal of Monetary Economics, Elsevier, vol. 54(1), pages 205-224, January.
- Annamaria Lusardi & Olivia S. Mitchell, 2006. "Baby Boomer Retirement Security: The Roles of Planning, Financial Literacy, and Housing Wealth," CeRP Working Papers 54, Center for Research on Pensions and Welfare Policies, Turin (Italy).
- Annamaria Lusardi & Olivia S. Mitchell, 2006. "Baby Boomer Retirement Security: the Roles of Planning, Financial Literacy, and Housing Wealth," NBER Working Papers 12585, National Bureau of Economic Research, Inc.
- Annamaria Lusardi & Olivia S. Mitchell, 2006. "Baby Boomer Retirement Security: The Roles of Planning, Financial Literacy, and Housing Wealth," Working Papers wp114, University of Michigan, Michigan Retirement Research Center.
- Lusardi, Annamaria & Mitchell, Olivia S., 2006. "Baby boomer retirement security: The roles of planning, financial literacy, and Housing wealth," CFS Working Paper Series 2006/20, Center for Financial Studies (CFS).
- Hazel Bateman & Christine Eckert & John Geweke & Jordan Louviere & Susan Thorp & Stephen Satchell, 2012. "Financial Competence and Expectations Formation: Evidence from Australia," The Economic Record, The Economic Society of Australia, vol. 88(280), pages 39-63, March.
- Annamaria Lusardi & Olivia S. Mitchell, 2017.
"How Ordinary Consumers Make Complex Economic Decisions: Financial Literacy and Retirement Readiness,"
Quarterly Journal of Finance (QJF), World Scientific Publishing Co. Pte. Ltd., vol. 7(03), pages 1-31, September.
- Annamaria Lusardi & Olivia S. Mitchell, 2009. "How Ordinary Consumers Make Complex Economic Decisions: Financial Literacy and Retirement Readiness," NBER Working Papers 15350, National Bureau of Economic Research, Inc.
- Lusardi, Annamaria & Mitchell, Olivia S., 2010. "How ordinary consumers make complex economic decisions: Financial literacy and retirement readiness," CFS Working Paper Series 2010/11, Center for Financial Studies (CFS).
- Annamaria Lusardi & Olivia S. Mitchell, 2009. "How Ordinary Consumers Make Complex Economic Decisions: Financial Literacy and Retirement Readiness," CeRP Working Papers 90, Center for Research on Pensions and Welfare Policies, Turin (Italy).
- Uri Gneezy & Jan Potters, 1997.
"An Experiment on Risk Taking and Evaluation Periods,"
The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 112(2), pages 631-645.
- Gneezy, U. & Potters, J.J.M., 1996. "An experiment on risk taking and evaluation periods," Other publications TiSEM e48d1f57-1bac-4a83-8772-c, Tilburg University, School of Economics and Management.
- Gneezy, U. & Potters, J.J.M., 1997. "An experiment on risk taking and evaluation periods," Other publications TiSEM da6ba1bf-e15c-41b2-ae95-c, Tilburg University, School of Economics and Management.
- Gneezy, U. & Potters, J.J.M., 1996. "An experiment on risk taking and evaluation periods," Discussion Paper 1996-61, Tilburg University, Center for Economic Research.
- Catherine C. Eckel & Philip J. Grossman, 2008. "Forecasting Risk Attitudes: An Experimental Study Using Actual and Forecast Gamble Choices," Monash Economics Working Papers archive-01, Monash University, Department of Economics.
- Charness, Gary & Gneezy, Uri & Imas, Alex, 2013. "Experimental methods: Eliciting risk preferences," Journal of Economic Behavior & Organization, Elsevier, vol. 87(C), pages 43-51.
- Charness, Gary & Karni, Edi & Levin, Dan, 2010.
"On the conjunction fallacy in probability judgment: New experimental evidence regarding Linda,"
Games and Economic Behavior, Elsevier, vol. 68(2), pages 551-556, March.
- Edi Karni, 2009. "On the Conjunction Fallacy in Probability Judgment: New Experimental Evidence Regarding Linda," Economics Working Paper Archive 552, The Johns Hopkins University,Department of Economics.
- Kaufman, Carol Felker & Lane, Paul M & Lindquist, Jay D, 1991. "Exploring More Than 24 Hours a Day: A Preliminary Investigation of Polychronic Time Use," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 18(3), pages 392-401, December.
- Reilly, Michael D, 1982. "Working Wives and Convenience Consumption," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 8(4), pages 407-418, March.
- David G. Rand & Alexander Peysakhovich & Gordon T. Kraft-Todd & George E. Newman & Owen Wurzbacher & Martin A. Nowak & Joshua D. Greene, 2014. "Social heuristics shape intuitive cooperation," Nature Communications, Nature, vol. 5(1), pages 1-12, May.
- Berinsky, Adam J. & Huber, Gregory A. & Lenz, Gabriel S., 2012. "Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk," Political Analysis, Cambridge University Press, vol. 20(3), pages 351-368, July.
Citations
Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
Cited by:
- Antonio A. Arechar & Gordon T. Kraft-Todd & David G. Rand, 2017. "Turking overtime: how participant characteristics and behavior vary over time and day on Amazon Mechanical Turk," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 3(1), pages 1-11, July.
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Jeanette A.M.J. Deetlefs & Mathew Chylinski & Andreas Ortmann, 2015. "MTurk ‘Unscrubbed’: Exploring the good, the ‘Super’, and the unreliable on Amazon’s Mechanical Turk," Discussion Papers 2015-20, School of Economics, The University of New South Wales.
- Collier, Trevor & Cotten, Stephen & Roush, Justin, 2022. "Using pandemic behavior to test the external validity of laboratory measurements of risk aversion and guilt," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 101(C).
- Wladislaw Mill & Cornelius Schneider, 2023. "The Bright Side of Tax Evasion," CESifo Working Paper Series 10615, CESifo.
- Ranganathan, Kavitha & Lejarraga, Tomás, 2021. "Elicitation of risk preferences through satisficing," Journal of Behavioral and Experimental Finance, Elsevier, vol. 32(C).
- Lusardi, Annamaria & Mitchell, Olivia S., 2011.
"Financial literacy around the world: an overview,"
Journal of Pension Economics and Finance, Cambridge University Press, vol. 10(4), pages 497-508, October.
- Annamaria Lusardi & Olivia S. Mitchell, 2011. "Financial Literacy Around the World: An Overview," CeRP Working Papers 106, Center for Research on Pensions and Welfare Policies, Turin (Italy).
- Annamaria Lusardi & Olivia S. Mitchell, 2011. "Financial Literacy around the World: An Overview," NBER Working Papers 17107, National Bureau of Economic Research, Inc.
- repec:cup:judgdm:v:14:y:2019:i:3:p:234-279 is not listed on IDEAS
- Prissé, Benjamin & Jorrat, Diego, 2022.
"Lab vs online experiments: No differences,"
Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 100(C).
- Benjamin Prissé & Diego Jorrat, 2022. "Lab vs online experiments: no differences," Working Papers 137, Red Nacional de Investigadores en Economía (RedNIE).
- Lefgren, Lars J. & Sims, David P. & Stoddard, Olga B., 2016. "Effort, luck, and voting for redistribution," Journal of Public Economics, Elsevier, vol. 143(C), pages 89-97.
- Joanne W. Hsu, 2016.
"Aging and Strategic Learning: The Impact of Spousal Incentives on Financial Literacy,"
Journal of Human Resources, University of Wisconsin Press, vol. 51(4), pages 1036-1067.
- Joanne W. Hsu, 2011. "Aging and Strategic Learning: The Impact of Spousal Incentives on Financial Literacy," NFI Working Papers 2011-WP-06, Indiana State University, Scott College of Business, Networks Financial Institute.
- Joanne W. Hsu, 2011. "Aging and strategic learning: the impact of spousal incentives on financial literacy," Finance and Economics Discussion Series 2011-53, Board of Governors of the Federal Reserve System (U.S.).
- Shin, Su Hyun & Kim, Kyoung Tae & Seay, Martin, 2020. "Sources of information and portfolio allocation," Journal of Economic Psychology, Elsevier, vol. 76(C).
- Sholevar, Maryam & Harris, Laurence, 2019. "Mind the gap: A discussion paper on Financial Literacy, Financial behaviour and Financial Education : Is there any Gender Gap?," OSF Preprints b7zd6, Center for Open Science.
- Oscar A. Stolper & Andreas Walter, 2017. "Financial literacy, financial advice, and financial behavior," Journal of Business Economics, Springer, vol. 87(5), pages 581-643, July.
- Baeckström, Ylva & Marsh, Ian W. & Silvester, Joanne, 2021. "Variations in investment advice provision: A study of financial advisors of millionaire investors," Journal of Economic Behavior & Organization, Elsevier, vol. 188(C), pages 716-735.
- Annamaria Lusardi & Olivia S. Mitchell & Vilsa Curto, 2009. "Financial Literacy and Financial Sophistication Among Older Americans," NBER Working Papers 15469, National Bureau of Economic Research, Inc.
- Caliendo, Frank N. & Findley, T. Scott, 2013. "Time inconsistency and retirement planning," Economics Letters, Elsevier, vol. 121(1), pages 30-34.
- Di Girolamo, Amalia & Harrison, Glenn W. & Lau, Morten I. & Swarthout, J. Todd, 2015.
"Subjective belief distributions and the characterization of economic literacy,"
Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 59(C), pages 1-12.
- Amalia Di Girolamo & Glenn W. Harrison & Morten I. Lau & J. Todd Swarthout, 2015. "Subjective Belief Distributions and the Characterization of Economic Literacy," Experimental Economics Center Working Paper Series 2015-06, Experimental Economics Center, Andrew Young School of Policy Studies, Georgia State University.
- Di Girolamo, Amalia & Harrison, Glenn & Lau, Morten & Swarthout, J. Todd, 2016. "Subjective Belief Distributions and the Characterization of Economic Literacy," IZA Discussion Papers 9856, Institute of Labor Economics (IZA).
- Sascha Füllbrunn & Wolfgang J. Luhan, 2015. "Am I my Peer‘s Keeper? Social Responsibility in Financial Decision Making," Ruhr Economic Papers 0551, Rheinisch-Westfälisches Institut für Wirtschaftsforschung, Ruhr-Universität Bochum, Universität Dortmund, Universität Duisburg-Essen.
- Gary Charness & Thomas Garcia & Theo Offerman & Marie Claire Villeval, 2020.
"Do measures of risk attitude in the laboratory predict behavior under risk in and outside of the laboratory?,"
Journal of Risk and Uncertainty, Springer, vol. 60(2), pages 99-123, April.
- Gary Charness & Thomas Garcia & Theo Offerman & Marie Claire Villeval, 2019. "Do measures of risk attitude in the laboratory predict behavior under risk in and outside of the laboratory?," Working Papers halshs-02146618, HAL.
- Gary Charness & Thomas Garcia & Theo Offerman & Marie Claire Villeval, 2020. "Do measures of risk attitude in the laboratory predict behavior under risk in and outside of the laboratory?," Post-Print halshs-02908109, HAL.
- Charness, Gary & Garcia, Thomas & Offerman, Theo & Villeval, Marie Claire, 2019. "Do Measures of Risk Attitude in the Laboratory Predict Behavior under Risk in and outside of the Laboratory?," IZA Discussion Papers 12395, Institute of Labor Economics (IZA).
- Gary Charness & Thomas Garcia & Theo Offerman & Marie Claire Villeval, 2019. "Do measures of risk attitude in the laboratory predict behavior under risk in and outside of the laboratory?," Working Papers 1921, Groupe d'Analyse et de Théorie Economique Lyon St-Étienne (GATE Lyon St-Étienne), Université de Lyon.
- Anderson, Anders & Baker, Forest & Robinson, David T., 2017.
"Precautionary savings, retirement planning and misperceptions of financial literacy,"
Journal of Financial Economics, Elsevier, vol. 126(2), pages 383-398.
- Anders Anderson & Forest Baker & David T. Robinson, 2015. "Precautionary Savings, Retirement Planning and Misperceptions of Financial Literacy," NBER Working Papers 21356, National Bureau of Economic Research, Inc.
- Tabea Bucher-Koenen & Annamaria Lusardi & Rob Alessie & Maarten van Rooij, 2017.
"How Financially Literate Are Women? An Overview and New Insights,"
Journal of Consumer Affairs, Wiley Blackwell, vol. 51(2), pages 255-283, July.
- Tabea Bucher-Koenen & Annamaria Lusardi & Rob Alessie & Maarten van Rooij, 2014. "How financially literate are women? An overview and new insights," NBER Working Papers 20793, National Bureau of Economic Research, Inc.
- Bucher-Koenen, Tabea & Lusardi, Annamaria & Alessie, Rob J. M. & Van Rooij, Maarten C. J., 2014. "How Financially Literate are Women? An Overview and New Insights," MEA discussion paper series 201419, Munich Center for the Economics of Aging (MEA) at the Max Planck Institute for Social Law and Social Policy.
- Murong Yang & Laurence S. J. Roope & James Buchanan & Arthur E. Attema & Philip M. Clarke & A. Sarah Walker & Sarah Wordsworth, 2022. "Eliciting risk preferences that predict risky health behavior: A comparison of two approaches," Health Economics, John Wiley & Sons, Ltd., vol. 31(5), pages 836-858, May.
More about this item
Keywords
data collection; experimentation; field experiment; internet; Mechanical Turk;All these keywords.
JEL classification:
- C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior
- C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
- D80 - Microeconomics - - Information, Knowledge, and Uncertainty - - - General
NEP fields
This paper has been announced in the following NEP Reports:- NEP-EXP-2015-10-04 (Experimental Economics)
Statistics
Access and download statisticsCorrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:swe:wpaper:2015-20. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Hongyi Li (email available below). General contact details of provider: https://edirc.repec.org/data/senswau.html .
Please note that corrections may take a couple of weeks to filter through the various RePEc services.