IDEAS home Printed from https://ideas.repec.org/a/sae/entthe/v48y2024i2p742-757.html

Test-Retest Reliability in Metric Conjoint Experiments: A New Workflow to Evaluate Confidence in Model Results

Author

Listed:
  • Jens Schüler
  • Brian S. Anderson
  • Charles Y. Murnieks
  • Matthias Baum
  • Alexander Küsshauer

Abstract

Metric conjoint studies are a popular research design in the entrepreneurship domain. For these studies, test-retest reliabilities of Ï â€‰> .70 or higher are an often-cited reliability criterion. Despite their widespread use, however, there is little rigorous analysis of whether test-retest reliability in metric conjoint studies relates to model efficacy. Informed by a systematic literature review, we conducted two Monte Carlo simulations to evaluate the effects of various determinants of test-retest reliability in conjoint experiments. We then illustrate a workflow for entrepreneurship researchers employing conjoint designs to better evaluate—and communicate—confidence in statistical models estimated from conjoint data.

Suggested Citation

  • Jens Schüler & Brian S. Anderson & Charles Y. Murnieks & Matthias Baum & Alexander Küsshauer, 2024. "Test-Retest Reliability in Metric Conjoint Experiments: A New Workflow to Evaluate Confidence in Model Results," Entrepreneurship Theory and Practice, , vol. 48(2), pages 742-757, March.
    • Handle: RePEc:sae:entthe:v:48:y:2024:i:2:p:742-757
    DOI: 10.1177/10422587231184071
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/10422587231184071
    Download Restriction: no
    File URL: https://libkey.io/10.1177/10422587231184071?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. John Antonakis & Samuel Bendahan & Philippe Jacquart & Rafael Lalive, 2010. "On making causal claims : A review and recommendations," Post-Print hal-02313119, HAL.
    2. Hyslop, Dean R & Imbens, Guido W, 2001. "Bias from Classical and Other Forms of Measurement Error," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(4), pages 475-481, October.
    3. Lurie, Nicholas H, 2004. "Decision Making in Information-Rich Environments: The Role of Information Structure," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 30(4), pages 473-486, March.
    4. Moser, Kilian J. & Tumasjan, Andranik & Welpe, Isabell M., 2017. "Small but attractive: Dimensions of new venture employer attractiveness and the moderating role of applicants' entrepreneurial behaviors," Journal of Business Venturing, Elsevier, vol. 32(5), pages 588-610.
    5. Louis Guttman, 1945. "A basis for analyzing test-retest reliability," Psychometrika, Springer;The Psychometric Society, vol. 10(4), pages 255-282, December.
    6. Markku Maula & Wouter Stam, 2020. "Enhancing Rigor in Quantitative Entrepreneurship Research," Entrepreneurship Theory and Practice, , vol. 44(6), pages 1059-1090, November.
    7. Wennberg, Karl & Anderson, Brian S., 2020. "Editorial: Enhancing the exploration and communication of quantitative entrepreneurship research," Journal of Business Venturing, Elsevier, vol. 35(3).
    8. David Reibstein & John E. G. Bateson & William Boulding, 1988. "Conjoint Analysis Reliability: Empirical Findings," Marketing Science, INFORMS, vol. 7(3), pages 271-286.
    9. David A. Hensher, 2006. "How do respondents process stated choice experiments? Attribute consideration under varying information load," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 21(6), pages 861-878.
    10. Green, Paul E & Srinivasan, V, 1978. "Conjoint Analysis in Consumer Research: Issues and Outlook," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 5(2), pages 103-123, Se.
    11. Grégoire, Denis A. & Binder, Julia K. & Rauch, Andreas, 2019. "Navigating the validity tradeoffs of entrepreneurship research experiments: A systematic review and best-practice suggestions," Journal of Business Venturing, Elsevier, vol. 34(2), pages 284-310.
    12. Busenitz, Lowell W. & Barney, Jay B., 1997. "Differences between entrepreneurs and managers in large organizations: Biases and heuristics in strategic decision-making," Journal of Business Venturing, Elsevier, vol. 12(1), pages 9-30, January.
    13. Brian S. Anderson & Jens Schueler & Matthias Baum & William J. Wales & Vishal K. Gupta, 2022. "The Chicken or the Egg? Causal Inference in Entrepreneurial Orientation–Performance Research," Entrepreneurship Theory and Practice, , vol. 46(6), pages 1569-1596, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sumaya Islam & Tobias Buchta & Colin Wooldridge, 2026. "Can You Have It All? Trade-Offs, Tensions, And The Esg Tightrope In Entrepreneurial Decision-Making," Working Papers Dissertations 169, Paderborn University, Faculty of Business Administration and Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Newbert, Scott L. & Kher, Romi & Yang, Shu, 2022. "Now that's interesting and important! Moving beyond averages to increase the inferential value of empirical findings in entrepreneurship research," Journal of Business Venturing, Elsevier, vol. 37(2).
    2. Küsshauer, Alexander & Baum, Matthias, 2023. "The good, the bad and the uncertain: Employers' perceptions of former entrepreneurs," Journal of Business Venturing, Elsevier, vol. 38(2).
    3. Philipp Ego & Ulrich Pidun, 2025. "Roles of Market Characteristics and Experience in Ventures’ International Growth Mode Choice: A Conjoint Experiment," Management International Review, Springer, vol. 65(2), pages 329-357, April.
    4. Teichert, Thorsten Andreas, 1997. "Schätzgenauigkeit von Conjoint-Analysen," Manuskripte aus den Instituten für Betriebswirtschaftslehre der Universität Kiel 444, Christian-Albrechts-Universität zu Kiel, Institut für Betriebswirtschaftslehre.
    5. Sylvia Hubner & Fabian Most & Jochen Wirtz & Christine Auer, 2022. "Narratives in entrepreneurial ecosystems: drivers of effectuation versus causation," Small Business Economics, Springer, vol. 59(1), pages 211-242, June.
    6. Fraser, Iain & Balcombe, Kelvin & Williams, Louis & McSorley, Eugene, 2021. "Preference stability in discrete choice experiments. Some evidence using eye-tracking," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 94(C).
    7. Seung Hoon D. Chung & Simon C. Parker, 2023. "Founder affiliations: jobseeker reactions and impact on employee recruitment by start-up ventures," Small Business Economics, Springer, vol. 61(1), pages 259-283, June.
    8. Olivier Toubia & Duncan I. Simester & John R. Hauser & Ely Dahan, 2003. "Fast Polyhedral Adaptive Conjoint Estimation," Marketing Science, INFORMS, vol. 22(3), pages 273-303.
    9. Brian S. Anderson & Jens Schueler & Matthias Baum & William J. Wales & Vishal K. Gupta, 2022. "The Chicken or the Egg? Causal Inference in Entrepreneurial Orientation–Performance Research," Entrepreneurship Theory and Practice, , vol. 46(6), pages 1569-1596, November.
    10. Markku Maula & Wouter Stam, 2020. "Enhancing Rigor in Quantitative Entrepreneurship Research," Entrepreneurship Theory and Practice, , vol. 44(6), pages 1059-1090, November.
    11. Elhem Ben Fatma & Léo-Paul Dana & Souhir Elleuch & Ezzeddine Ben Mohamed, 2024. "Does dispositional optimism affect entrepreneurial success? Evidence from Saudi Arabia," International Entrepreneurship and Management Journal, Springer, vol. 20(2), pages 1127-1152, June.
    12. Marie Madeleine Meurer & Maksim Belitski & Christian Fisch & Roy Thurik, 2024. "What gets published and what doesn’t? Exploring optimal distinctiveness and diverse expectations in entrepreneurship articles," Small Business Economics, Springer, vol. 63(3), pages 1139-1170, October.
    13. Douglas, Evan J., 2013. "Reconstructing entrepreneurial intentions to identify predisposition for growth," Journal of Business Venturing, Elsevier, vol. 28(5), pages 633-651.
    14. Brinkerink, Jasper & De Massis, Alfredo & Kellermanns, Franz, 2022. "One finding is no finding: Toward a replication culture in family business research," Journal of Family Business Strategy, Elsevier, vol. 13(4).
    15. Montgomery, David B. & Ramus, Catherine, 2007. "Including Corporate Social Responsibility, Environmental Sustainaibility, and Ethics in Calibrating MBA Job Preferences," Research Papers 1981, Stanford University, Graduate School of Business.
    16. Stefanie Weniger & Svenja Jarchow, 2023. "Between benefit and risk: how entrepreneurs evaluate corporate investors," Journal of Business Economics, Springer, vol. 93(5), pages 783-816, July.
    17. Morgan, Horatio M. & Sui, Sui & Baum, Matthias, 2018. "Are SMEs with immigrant owners exceptional exporters?," Journal of Business Venturing, Elsevier, vol. 33(3), pages 241-260.
    18. Gala, Kaushik & Schwab, Andreas, 2024. "Stars everywhere: Revealing the prevalence of star performers using empirical data published in entrepreneurship research," Journal of Business Venturing Insights, Elsevier, vol. 22(C).
    19. Hähnlein, Johannes & Baum, Matthias & Durst, Carolin, 2025. "Cultivating the ecosystem: How social exchange sows the seeds of entrepreneurial contributions," Journal of Business Venturing Insights, Elsevier, vol. 24(C).
    20. Saulo Dubard Barbosa & Brett R. Smith, 2024. "Specifying the role of religion in entrepreneurial action: a cognitive perspective," Small Business Economics, Springer, vol. 62(4), pages 1315-1336, April.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:entthe:v:48:y:2024:i:2:p:742-757. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.