IDEAS home Printed from https://ideas.repec.org/a/oup/jconrs/v44y2018i5p1157-1173..html

Increasing the Power of Your Study by Increasing the Effect Size

Author

Listed:
  • Tom Meyvis
  • Stijn M J Van Osselaer
  • Dahl DarrenEditor
  • Eileen FischerEditor
  • Gita JoharEditor
  • Vicki MorwitzEditor

Abstract

As in other social sciences, published findings in consumer research tend to overestimate the size of the effect being investigated, due to both file drawer effects and abuse of researcher degrees of freedom, including opportunistic analysis decisions. Given that most effect sizes are substantially smaller than would be apparent from published research, there has been a widespread call to increase power by increasing sample size. We propose that, aside from increasing sample size, researchers can also increase power by boosting the effect size. If done correctly, removing participants, using covariates, and optimizing experimental designs, stimuli, and measures can boost effect size without inflating researcher degrees of freedom. In fact, careful planning of studies and analyses to maximize effect size is essential to be able to study many psychologically interesting phenomena when massive sample sizes are not feasible.

Suggested Citation

  • Tom Meyvis & Stijn M J Van Osselaer & Dahl DarrenEditor & Eileen FischerEditor & Gita JoharEditor & Vicki MorwitzEditor, 2018. "Increasing the Power of Your Study by Increasing the Effect Size," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 44(5), pages 1157-1173.
    • Handle: RePEc:oup:jconrs:v:44:y:2018:i:5:p:1157-1173.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/jcr/ucx110
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Neil Stewart & Christoph Ungemach & Adam J. L. Harris & Daniel M. Bartels & Ben R. Newell & Gabriele Paolacci & Jesse Chandler, "undated". "The Average Laboratory Samples a Population of 7,300 Amazon Mechanical Turk Workers," Mathematica Policy Research Reports f97b669c7b3e4c2ab95c9f805, Mathematica Policy Research.
    2. Sawyer, Alan G, 1975. "Demand Artifacts in Laboratory Experiments in Consumer Research," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 1(4), pages 20-30, March.
    3. Neil Stewart & Christoph Ungemach & Adam J. L. Harris & Daniel M. Bartels & Ben R. Newell & Gabriele Paolacci & Jesse Chandler, 2015. "The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 10(5), pages 479-491, September.
    4. Joseph K. Goodman & Gabriele Paolacci, 2017. "Crowdsourcing Consumer Research," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 44(1), pages 196-210.
    5. Novemsky, Nathan & Ratner, Rebecca K, 2003. "The Time Course and Impact of Consumers' Erroneous Beliefs about Hedonic Contrast Effects," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 29(4), pages 507-516, March.
    6. Kathryn Sharpe Wessling & Joel Huber & Oded Netzer, 2017. "MTurk Character Misrepresentation: Assessment and Solutions," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 44(1), pages 211-230.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Austin M Strange & Ryan D Enos & Mark Hill & Amy Lakeman, 2019. "Online volunteer laboratories for human subjects research," PLOS ONE, Public Library of Science, vol. 14(8), pages 1-13, August.
    2. Xiaochuan Song & Graham H. Lowman & Peter Harms, 2020. "Justice for the Crowd: Organizational Justice and Turnover in Crowd-Based Labor," Administrative Sciences, MDPI, vol. 10(4), pages 1-37, November.
    3. Cloos, Janis & Greiff, Matthias & Rusch, Hannes, 2020. "Geographical Concentration and Editorial Favoritism within the Field of Laboratory Experimental Economics (RM/19/029-revised-)," Research Memorandum 014, Maastricht University, Graduate School of Business and Economics (GSBE).
    4. John Hulland & Jeff Miller, 2018. "“Keep on Turkin’”?," Journal of the Academy of Marketing Science, Springer, vol. 46(5), pages 789-794, September.
    5. Roman Lukyanenko & Andrea Wiggins & Holly K. Rosser, 0. "Citizen Science: An Information Quality Research Frontier," Information Systems Frontiers, Springer, vol. 0, pages 1-23.
    6. Clément Le Ludec & Paola Tubaro & Antonio A. Casilli, 2019. "How many people microwork in France? Estimating the size of a new labor force," Working Papers hal-02012731, HAL.
    7. Martínez-López, Francisco J. & Li, Yangchun & Su, Wan & Feng, Changyuan, 2019. "To have or have not: Buy buttons on social platforms," Journal of Business Research, Elsevier, vol. 105(C), pages 33-48.
    8. David Ronayne & Daniel Sgroi, 2018. "On the motivations for the dual-use of electronic and traditional cigarettes," Applied Economics Letters, Taylor & Francis Journals, vol. 25(12), pages 830-834, July.
    9. Christ, Margaret H. & Vance, Thomas W., 2018. "Cascading controls: The effects of managers’ incentives on subordinate effort to help or harm," Accounting, Organizations and Society, Elsevier, vol. 65(C), pages 20-32.
    10. Antonio A. Arechar & Simon Gächter & Lucas Molleman, 2018. "Conducting interactive experiments online," Experimental Economics, Springer;Economic Science Association, vol. 21(1), pages 99-131, March.
    11. Cloos, Janis & Greiff, Matthias & Rusch, Hannes, 2019. "Geographical Concentration and Editorial Favoritism within the Field of Laboratory Experimental Economics," Research Memorandum 029, Maastricht University, Graduate School of Business and Economics (GSBE).
    12. Marta Pizzetti & Lucia Gatti & Peter Seele, 2021. "Firms Talk, Suppliers Walk: Analyzing the Locus of Greenwashing in the Blame Game and Introducing ‘Vicarious Greenwashing’," Journal of Business Ethics, Springer, vol. 170(1), pages 21-38, April.
    13. Kevin C Elliott & Aaron M McCright & Summer Allen & Thomas Dietz, 2017. "Values in environmental research: Citizens’ views of scientists who acknowledge values," PLOS ONE, Public Library of Science, vol. 12(10), pages 1-18, October.
    14. Shuili Du & Mayowa T. Babalola & Premilla D’Cruz & Edina Dóci & Lucia Garcia-Lorenzo & Louise Hassan & Gazi Islam & Alexander Newman & Ernesto Noronha & Suzanne Gils, 2024. "The Ethical, Societal, and Global Implications of Crowdsourcing Research," Journal of Business Ethics, Springer, vol. 193(1), pages 1-16, August.
    15. Palan, Stefan & Schitter, Christian, 2018. "Prolific.ac—A subject pool for online experiments," Journal of Behavioral and Experimental Finance, Elsevier, vol. 17(C), pages 22-27.
    16. Clément Le Ludec & Paola Tubaro & Antonio A. Casilli, 2019. "Combien de personnes micro-travaillent en France ? Estimer l'ampleur d'une nouvelle forme de travail," Working Papers hal-02021525, HAL.
    17. Arndt, Aaron D. & Ford, John B. & Babin, Barry J. & Luong, Vinh, 2022. "Collecting samples from online services: How to use screeners to improve data quality," International Journal of Research in Marketing, Elsevier, vol. 39(1), pages 117-133.
    18. Sergio Alessandro Castagnetti & Sebastiano Massaro & Eugenio Proto, 2021. "The Influence of Anger on Strategic Cooperative Interactions," Working Papers 2021_05, Business School - Economics, University of Glasgow.
    19. Jiachun Lu & Lutz Kaufmann & Craig R. Carter, 2021. "How Informal Exchanges Impact Formal Sourcing Collaboration (and What Supply Managers Can Do about It)," Journal of Supply Chain Management, Institute for Supply Management, vol. 57(4), pages 26-62, October.
    20. Logan S. Casey & Jesse Chandler & Adam Seth Levine & Andrew Proctor & Dara Z. Strolovitch, 2017. "Intertemporal Differences Among MTurk Workers: Time-Based Sample Variations and Implications for Online Data Collection," SAGE Open, , vol. 7(2), pages 21582440177, June.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:jconrs:v:44:y:2018:i:5:p:1157-1173.. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://academic.oup.com/jcr .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.