IDEAS home Printed from https://ideas.repec.org/a/oup/jconrs/v44y2018i5p1157-1173..html
   My bibliography  Save this article

Increasing the Power of Your Study by Increasing the Effect Size

Author

Listed:
  • Tom Meyvis
  • Stijn M J Van Osselaer
  • Dahl DarrenEditor
  • Eileen FischerEditor
  • Gita JoharEditor
  • Vicki MorwitzEditor

Abstract

As in other social sciences, published findings in consumer research tend to overestimate the size of the effect being investigated, due to both file drawer effects and abuse of researcher degrees of freedom, including opportunistic analysis decisions. Given that most effect sizes are substantially smaller than would be apparent from published research, there has been a widespread call to increase power by increasing sample size. We propose that, aside from increasing sample size, researchers can also increase power by boosting the effect size. If done correctly, removing participants, using covariates, and optimizing experimental designs, stimuli, and measures can boost effect size without inflating researcher degrees of freedom. In fact, careful planning of studies and analyses to maximize effect size is essential to be able to study many psychologically interesting phenomena when massive sample sizes are not feasible.

Suggested Citation

  • Tom Meyvis & Stijn M J Van Osselaer & Dahl DarrenEditor & Eileen FischerEditor & Gita JoharEditor & Vicki MorwitzEditor, 2018. "Increasing the Power of Your Study by Increasing the Effect Size," Journal of Consumer Research, Oxford University Press, vol. 44(5), pages 1157-1173.
  • Handle: RePEc:oup:jconrs:v:44:y:2018:i:5:p:1157-1173.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/jcr/ucx110
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Neil Stewart & Christoph Ungemach & Adam J. L. Harris & Daniel M. Bartels & Ben R. Newell & Gabriele Paolacci & Jesse Chandler, "undated". "The Average Laboratory Samples a Population of 7,300 Amazon Mechanical Turk Workers," Mathematica Policy Research Reports f97b669c7b3e4c2ab95c9f805, Mathematica Policy Research.
    2. Sawyer, Alan G, 1975. "Demand Artifacts in Laboratory Experiments in Consumer Research," Journal of Consumer Research, Oxford University Press, vol. 1(4), pages 20-30, March.
    3. Joseph K. Goodman & Gabriele Paolacci, 2017. "Crowdsourcing Consumer Research," Journal of Consumer Research, Oxford University Press, vol. 44(1), pages 196-210.
    4. repec:cup:judgdm:v:10:y:2015:i:5:p:479-491 is not listed on IDEAS
    5. Novemsky, Nathan & Ratner, Rebecca K, 2003. "The Time Course and Impact of Consumers' Erroneous Beliefs about Hedonic Contrast Effects," Journal of Consumer Research, Oxford University Press, vol. 29(4), pages 507-516, March.
    6. Kathryn Sharpe Wessling & Joel Huber & Oded Netzer, 2017. "MTurk Character Misrepresentation: Assessment and Solutions," Journal of Consumer Research, Oxford University Press, vol. 44(1), pages 211-230.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Antonia Krefeld-Schwalb & Benjamin Scheibehenne, 2023. "Tighter nets for smaller fishes? Mapping the development of statistical practices in consumer research between 2008 and 2020," Marketing Letters, Springer, vol. 34(3), pages 351-365, September.
    2. Wang, Lili & Lyu, Jiaying, 2019. "Inspiring awe through tourism and its consequence," Annals of Tourism Research, Elsevier, vol. 77(C), pages 106-116.
    3. Ahmad, Fayez & Guzmán, Francisco & Kidwell, Blair, 2022. "Effective messaging strategies to increase brand love for sociopolitical activist brands," Journal of Business Research, Elsevier, vol. 151(C), pages 609-622.
    4. Wolf, Tobias & Jahn, Steffen & Hammerschmidt, Maik & Weiger, Welf H., 2021. "Competition versus cooperation: How technology-facilitated social interdependence initiates the self-improvement chain," International Journal of Research in Marketing, Elsevier, vol. 38(2), pages 472-491.
    5. Krefeld-Schwalb, Antonia & Rosner, Agnes, 2020. "A new way to guide consumer's choice: Retro-cueing alters the availability of product information in memory," Journal of Business Research, Elsevier, vol. 111(C), pages 135-147.
    6. Lasarov, Wassili & Mai, Robert & Hoffmann, Stefan, 2022. "The backfire effect of sustainable social cues. New evidence on social moral licensing," Ecological Economics, Elsevier, vol. 195(C).
    7. Xu, Lina & Roy, Abhijit, 2022. "Cashback as cash forward: The serial mediating effect of time/effort and money savings," Journal of Business Research, Elsevier, vol. 149(C), pages 30-37.
    8. Pan, Li (Sunny) & Pezzuti, Todd & Lu, Wei & Pechmann, Cornelia (Connie), 2019. "Hyperopia and frugality: Different motivational drivers and yet similar effects on consumer spending," Journal of Business Research, Elsevier, vol. 95(C), pages 347-356.
    9. Lembregts, Christophe & Cadario, Romain, 2024. "Consumer-Driven Climate Mitigation: Exploring Barriers and Solutions in Studying Higher Mitigation Potential Behaviors," OSF Preprints ywus6, Center for Open Science.
    10. Amit, Elinor & Danziger, Shai & Smith, Pamela K., 2022. "Medium is a powerful message: Pictures signal less power than words," Organizational Behavior and Human Decision Processes, Elsevier, vol. 169(C).
    11. Aghakhani, Hamed & Main, Kelley J., 2019. "Can two negatives make a positive? Social exclusion prevents carryover effects from deceptive advertising," Journal of Retailing and Consumer Services, Elsevier, vol. 47(C), pages 206-214.
    12. Johannes D. Hattula & Walter Herzog & Ravi Dhar, 2023. "The impact of touchscreen devices on consumers’ choice confidence and purchase likelihood," Marketing Letters, Springer, vol. 34(1), pages 35-53, March.
    13. John W. Gardner*, 2020. "Managing Production Yields and Rework through Feedback on Speed, Quality, and Quantity," Production and Operations Management, Production and Operations Management Society, vol. 29(9), pages 2182-2209, September.
    14. Simon J Blanchard & Jacob Goldenberg & Koen Pauwels & David A Schweidel, 2022. "Promoting Data Richness in Consumer Research: How to Develop and Evaluate Articles with Multiple Data Sources [The Critical Role of Methodological Pluralism for Policy-Relevant Empirical Marketing ," Journal of Consumer Research, Oxford University Press, vol. 49(2), pages 359-372.
    15. Otterbring, Tobias, 2021. "Peer presence promotes popular choices: A “Spicy†field study on social influence and brand choice," Journal of Retailing and Consumer Services, Elsevier, vol. 61(C).
    16. Jamel Khenfer & Steven Shepherd & Olivier Trendel, 2020. "Customer empowerment in the face of perceived Incompetence: Effect on preference for anthropomorphized brands," Grenoble Ecole de Management (Post-Print) hal-03189338, HAL.
    17. Jamel Khenfer & Steven Shepherd & Olivier Trendel, 2020. "Customer empowerment in the face of perceived Incompetence: Effect on preference for anthropomorphized brands," Post-Print hal-03189338, HAL.
    18. B. Claus & L. Warlop, 2022. "The Car Cushion Hypothesis: Bigger Cars Lead to More Risk Taking—Evidence from Behavioural Data," Journal of Consumer Policy, Springer, vol. 45(2), pages 331-342, June.
    19. Khenfer, Jamel & Shepherd, Steven & Trendel, Olivier, 2020. "Customer empowerment in the face of perceived Incompetence: Effect on preference for anthropomorphized brands," Journal of Business Research, Elsevier, vol. 118(C), pages 1-11.
    20. Choi, Jungsil & Li, Yexin Jessica & Rangan, Priyamvadha & Yin, Bingqing & Singh, Surendra N., 2020. "Opposites attract: Impact of background color on effectiveness of emotional charity appeals," International Journal of Research in Marketing, Elsevier, vol. 37(3), pages 644-660.
    21. Daryanto, Ahmad & Lukas, Bryan A., 2022. "Controlling for spurious moderation in marketing: A review of statistical techniques," Journal of Business Research, Elsevier, vol. 144(C), pages 180-192.
    22. Frank, Darius-Aurel & Otterbring, Tobias, 2023. "Being seen… by human or machine? Acknowledgment effects on customer responses differ between human and robotic service workers," Technological Forecasting and Social Change, Elsevier, vol. 189(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. John Hulland & Jeff Miller, 2018. "“Keep on Turkin’”?," Journal of the Academy of Marketing Science, Springer, vol. 46(5), pages 789-794, September.
    2. Martínez-López, Francisco J. & Li, Yangchun & Su, Wan & Feng, Changyuan, 2019. "To have or have not: Buy buttons on social platforms," Journal of Business Research, Elsevier, vol. 105(C), pages 33-48.
    3. Austin M Strange & Ryan D Enos & Mark Hill & Amy Lakeman, 2019. "Online volunteer laboratories for human subjects research," PLOS ONE, Public Library of Science, vol. 14(8), pages 1-13, August.
    4. Marta Pizzetti & Lucia Gatti & Peter Seele, 2021. "Firms Talk, Suppliers Walk: Analyzing the Locus of Greenwashing in the Blame Game and Introducing ‘Vicarious Greenwashing’," Journal of Business Ethics, Springer, vol. 170(1), pages 21-38, April.
    5. Arndt, Aaron D. & Ford, John B. & Babin, Barry J. & Luong, Vinh, 2022. "Collecting samples from online services: How to use screeners to improve data quality," International Journal of Research in Marketing, Elsevier, vol. 39(1), pages 117-133.
    6. Jiachun Lu & Lutz Kaufmann & Craig R. Carter, 2021. "How Informal Exchanges Impact Formal Sourcing Collaboration (and What Supply Managers Can Do about It)," Journal of Supply Chain Management, Institute for Supply Management, vol. 57(4), pages 26-62, October.
    7. Stephanie Anne Nicole Bedard & Carri Reisdorf Tolmie, 2018. "Millennials' green consumption behaviour: Exploring the role of social media," Corporate Social Responsibility and Environmental Management, John Wiley & Sons, vol. 25(6), pages 1388-1396, November.
    8. Orth, Ulrich R. & Hoffmann, Stefan & Nickel, Kristina, 2019. "Moral decoupling feels good and makes buying counterfeits easy," Journal of Business Research, Elsevier, vol. 98(C), pages 117-125.
    9. Kuchmaner, Christina A. & Wiggins, Jennifer & Grimm, Pamela E., 2019. "The Role of Network Embeddedness and Psychological Ownership in Consumer Responses to Brand Transgressions," Journal of Interactive Marketing, Elsevier, vol. 47(C), pages 129-143.
    10. Xiaochuan Song & Graham H. Lowman & Peter Harms, 2020. "Justice for the Crowd: Organizational Justice and Turnover in Crowd-Based Labor," Administrative Sciences, MDPI, vol. 10(4), pages 1-37, November.
    11. Kareklas, Ioannis & Muehling, Darrel D. & King, Skyler, 2019. "The effect of color and self-view priming in persuasive communications," Journal of Business Research, Elsevier, vol. 98(C), pages 33-49.
    12. Yu Ding & Wayne S. DeSarbo & Dominique M. Hanssens & Kamel Jedidi & John G. Lynch & Donald R. Lehmann, 2020. "The past, present, and future of measurement and methods in marketing analysis," Marketing Letters, Springer, vol. 31(2), pages 175-186, September.
    13. Mark Groza & Mya Pronschinske & Matthew Walker, 2011. "Perceived Organizational Motives and Consumer Responses to Proactive and Reactive CSR," Journal of Business Ethics, Springer, vol. 102(4), pages 639-652, September.
    14. Cloos, Janis & Greiff, Matthias & Rusch, Hannes, 2020. "Geographical Concentration and Editorial Favoritism within the Field of Laboratory Experimental Economics (RM/19/029-revised-)," Research Memorandum 014, Maastricht University, Graduate School of Business and Economics (GSBE).
    15. Azer, Jaylan & Anker, Thomas & Taheri, Babak & Tinsley, Ross, 2023. "Consumer-Driven racial stigmatization: The moderating role of race in online consumer-to-consumer reviews," Journal of Business Research, Elsevier, vol. 157(C).
    16. Deena A. Isom & Hunter M. Boehme & Toniqua C. Mikell & Stephen Chicoine & Marion Renner, 2021. "Status Threat, Social Concerns, and Conservative Media: A Look at White America and the Alt-Right," Societies, MDPI, vol. 11(3), pages 1-20, July.
    17. Eric T. Anderson & Duncan I. Simester, 2001. "Are Sale Signs Less Effective When More Products Have Them?," Marketing Science, INFORMS, vol. 20(2), pages 121-142, March.
    18. Lefebvre, Mathieu & Vieider, Ferdinand M. & Villeval, Marie Claire, 2010. "Incentive effects on risk attitude in small probability prospects," Economics Letters, Elsevier, vol. 109(2), pages 115-120, November.
    19. Benedikt Berger & Martin Adam & Alexander Rühr & Alexander Benlian, 2021. "Watch Me Improve—Algorithm Aversion and Demonstrating the Ability to Learn," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 63(1), pages 55-68, February.
    20. Robert Cialdini & Yexin Jessica Li & Adriana Samper & Ned Wellman, 2021. "How Bad Apples Promote Bad Barrels: Unethical Leader Behavior and the Selective Attrition Effect," Journal of Business Ethics, Springer, vol. 168(4), pages 861-880, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:jconrs:v:44:y:2018:i:5:p:1157-1173.. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://academic.oup.com/jcr .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.