IDEAS home Printed from https://ideas.repec.org/p/osf/osfxxx/xhe3u.html
   My bibliography  Save this paper

How many participants do i need to test an interaction? Conducting an appropriate power analysis and achieving sufficient power to detect an interaction

Author

Listed:
  • Sommet, Nicolas
  • Weissman, David Laurence

    (University of Rochester)

  • Cheutin, Nicolas
  • Elliot, Andrew

Abstract

Power analysis for first-order interactions poses two challenges: (i) Conducting an appropriate power analysis is difficult, because the required sample size depends on the shape of the interaction and the size of the simple slopes; (ii) Achieving sufficient power is difficult, because interactions tend to be modest in size. This paper consists of three parts. PART 1 addresses the first challenge. We first describe a fictional study to illustrate in a simple way how power analyses differ between interactions and main effects. Then, we introduce an intuitive taxonomy of 12 types of interactions based on the shape of the interaction (reversed, fully attenuated, partially attenuated) and the size of the simple effects (small, medium, large), and we offer mathematically-derived sample size recommendations to detect each interaction with a power of .80/.90. PART 2 addresses the second challenge. We first describe a preregistered meta-study (159 studies from recent articles in influential psychology journals) showing that the median power to detect medium-sized interactions in the literature is .18. Then, we use simulations (887M simulated datasets) to generate power curves for the 12 types of interactions, and test three strategies to increase power without increasing sample size: (i) preregistering one-tailed tests (+21% gain), (ii) using a mixed design (+75% gain), and (ii) preregistering contrast analysis for a fully attenuated interaction (+63% gain). PART 3 introduces INT×Power, a web-application that enables users to draw their interaction and determine the sample size needed to reach a power of .80 with and without using these strategies: www.intxpower.com.

Suggested Citation

  • Sommet, Nicolas & Weissman, David Laurence & Cheutin, Nicolas & Elliot, Andrew, 2022. "How many participants do i need to test an interaction? Conducting an appropriate power analysis and achieving sufficient power to detect an interaction," OSF Preprints xhe3u, Center for Open Science.
  • Handle: RePEc:osf:osfxxx:xhe3u
    DOI: 10.31219/osf.io/xhe3u
    as

    Download full text from publisher

    File URL: https://osf.io/download/6318ad7909a85221a3d9e41c/
    Download Restriction: no

    File URL: https://libkey.io/10.31219/osf.io/xhe3u?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Majer, Johann M. & Zhang, Kai & Zhang, Hong & Höhne, Benjamin P. & Trötschel, Roman, 2022. "Give and take frames in shared-resource negotiations," Journal of Economic Psychology, Elsevier, vol. 90(C).
    2. Larry V. Hedges & Jacob M. Schauer, 2019. "More Than One Replication Study Is Needed for Unambiguous Tests of Replication," Journal of Educational and Behavioral Statistics, , vol. 44(5), pages 543-570, October.
    3. Wynne W. Chin & Barbara L. Marcolin & Peter R. Newsted, 2003. "A Partial Least Squares Latent Variable Modeling Approach for Measuring Interaction Effects: Results from a Monte Carlo Simulation Study and an Electronic-Mail Emotion/Adoption Study," Information Systems Research, INFORMS, vol. 14(2), pages 189-217, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Liu, Qian & Shao, Zhen & Fan, Weiguo, 2018. "The impact of users’ sense of belonging on social media habit formation: Empirical evidence from social networking and microblogging websites in China," International Journal of Information Management, Elsevier, vol. 43(C), pages 209-223.
    2. Hazem Ali & Ting Chen & Yunhong Hao, 2021. "Sustainable Manufacturing Practices, Competitive Capabilities, and Sustainable Performance: Moderating Role of Environmental Regulations," Sustainability, MDPI, vol. 13(18), pages 1-19, September.
    3. Claudio Vitari & Elisabetta Raguseo, 2016. "Big data value and financial performance: an empirical investigation [Digital data, dynamic capability and financial performance: an empirical investigation in the era of Big Data]," Post-Print halshs-01923271, HAL.
    4. Mario Silic & Andrea Back, 2016. "The Influence of Risk Factors in Decision-Making Process for Open Source Software Adoption," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 15(01), pages 151-185, January.
    5. Gupta, Prashant & Seetharaman, A. & Raj, John Rudolph, 2013. "The usage and adoption of cloud computing by small and medium businesses," International Journal of Information Management, Elsevier, vol. 33(5), pages 861-874.
    6. Asif Khan & Chih-Cheng Chen & Kwanrat Suanpong & Athapol Ruangkanjanases & Santhaya Kittikowit & Shih-Chih Chen, 2021. "The Impact of CSR on Sustainable Innovation Ambidexterity: The Mediating Role of Sustainable Supply Chain Management and Second-Order Social Capital," Sustainability, MDPI, vol. 13(21), pages 1-25, November.
    7. Teller, Christoph & Kotzab, Herbert & Grant, David B., 2012. "The relevance of shopper logistics for consumers of store-based retail formats," Journal of Retailing and Consumer Services, Elsevier, vol. 19(1), pages 59-66.
    8. Mehwish Iftikhar & Muhammad Imran Qureshi & Shazia Qayyum & Iram Fatima & Sriyanto Sriyanto & Yasinta Indrianti & Aqeel Khan & Leo-Paul Dana, 2021. "Impact of Multifaceted Workplace Bullying on the Relationships between Technology Usage, Organisational Climate and Employee Physical and Emotional Health," IJERPH, MDPI, vol. 18(6), pages 1-19, March.
    9. Bulent Akkaya & Anna Bagieńska, 2022. "The Role of Agile Women Leadership in Achieving Team Effectiveness through Interpersonal Trust for Business Agility," Sustainability, MDPI, vol. 14(7), pages 1-16, March.
    10. Kawai, Norifumi & Chung, Chul, 2019. "Expatriate utilization, subsidiary knowledge creation and performance: The moderating role of subsidiary strategic context," Journal of World Business, Elsevier, vol. 54(1), pages 24-36.
    11. M Ashraf Al Haq & Norazlina Abd Wahab & Md. Mahmudul Alam, 2021. "Understanding The Impact Of Institutional Factors On Asnaf Sustainability: A Pls-Sem Approach," Journal of Islamic Monetary Economics and Finance, Bank Indonesia, vol. 7(4), pages 759-790, November.
    12. Riffat Ara Zannat Tama & Md Mahmudul Hoque & Ying Liu & Mohammad Jahangir Alam & Mark Yu, 2023. "An Application of Partial Least Squares Structural Equation Modeling (PLS-SEM) to Examining Farmers’ Behavioral Attitude and Intention towards Conservation Agriculture in Bangladesh," Agriculture, MDPI, vol. 13(2), pages 1-22, February.
    13. Osama Sohaib & Kyeong Kang & Mohammad Nurunnabi, 2018. "Gender-Based iTrust in E-Commerce: The Moderating Role of Cognitive Innovativeness," Sustainability, MDPI, vol. 11(1), pages 1-16, December.
    14. Zhang, Chu-Bing & Zhang, Zhuo-Ping & Chang, Ying & Li, Tian-Ge & Hou, Ru-Jing, 2022. "Effect of WeChat interaction on brand evaluation: A moderated mediation model of para-social interaction and affiliative tendency," Journal of Retailing and Consumer Services, Elsevier, vol. 64(C).
    15. Der-Chiang Li & Ching-Yeh Tsai, 2020. "Antecedents of Employees’ Goal Orientation and the Effects of Goal Orientation on E-Learning Outcomes: The Roles of Intra-Organizational Environment," Sustainability, MDPI, vol. 12(11), pages 1-27, June.
    16. Mosteller, Jill & Donthu, Naveen & Eroglu, Sevgin, 2014. "The fluent online shopping experience," Journal of Business Research, Elsevier, vol. 67(11), pages 2486-2493.
    17. Paul, Michael & Hennig-Thurau, Thorsten & Groth, Markus, 2015. "Tightening or loosening the “iron cage”? The impact of formal and informal display controls on service customers," Journal of Business Research, Elsevier, vol. 68(5), pages 1062-1073.
    18. Anis Ur Rehman & Mazhar Abbas & Faraz Ahmad Abbasi & Shoaib Khan, 2023. "How Tourist Experience Quality, Perceived Price Reasonableness and Regenerative Tourism Involvement Influence Tourist Satisfaction: A Study of Ha’il Region, Saudi Arabia," Sustainability, MDPI, vol. 15(2), pages 1-27, January.
    19. Yu Hao & Yingting Wang & Qiuwei Wu & Shiwei Sun & Weilu Wang & Menglin Cui, 2020. "What affects residents' participation in the circular economy for sustainable development? Evidence from China," Sustainable Development, John Wiley & Sons, Ltd., vol. 28(5), pages 1251-1268, September.
    20. Maggie Chuoyan Dong & Yulin Fang & Detmar W. Straub, 2017. "The Impact of Institutional Distance on the Joint Performance of Collaborating Firms: The Role of Adaptive Interorganizational Systems," Information Systems Research, INFORMS, vol. 28(2), pages 309-331, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:osfxxx:xhe3u. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.