IDEAS home Printed from https://ideas.repec.org/a/sae/entthe/v48y2024i2p742-757.html

Test-Retest Reliability in Metric Conjoint Experiments: A New Workflow to Evaluate Confidence in Model Results

Author

Listed:
  • Jens Schüler
  • Brian S. Anderson
  • Charles Y. Murnieks
  • Matthias Baum
  • Alexander Küsshauer

Abstract

Metric conjoint studies are a popular research design in the entrepreneurship domain. For these studies, test-retest reliabilities of Ï â€‰> .70 or higher are an often-cited reliability criterion. Despite their widespread use, however, there is little rigorous analysis of whether test-retest reliability in metric conjoint studies relates to model efficacy. Informed by a systematic literature review, we conducted two Monte Carlo simulations to evaluate the effects of various determinants of test-retest reliability in conjoint experiments. We then illustrate a workflow for entrepreneurship researchers employing conjoint designs to better evaluate—and communicate—confidence in statistical models estimated from conjoint data.

Suggested Citation

  • Jens Schüler & Brian S. Anderson & Charles Y. Murnieks & Matthias Baum & Alexander Küsshauer, 2024. "Test-Retest Reliability in Metric Conjoint Experiments: A New Workflow to Evaluate Confidence in Model Results," Entrepreneurship Theory and Practice, , vol. 48(2), pages 742-757, March.
    • Handle: RePEc:sae:entthe:v:48:y:2024:i:2:p:742-757
    DOI: 10.1177/10422587231184071
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/10422587231184071
    Download Restriction: no
    File URL: https://libkey.io/10.1177/10422587231184071?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. John Antonakis & Samuel Bendahan & Philippe Jacquart & Rafael Lalive, 2010. "On making causal claims : A review and recommendations," Post-Print hal-02313119, HAL.
    2. Hyslop, Dean R & Imbens, Guido W, 2001. "Bias from Classical and Other Forms of Measurement Error," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(4), pages 475-481, October.
    3. Lurie, Nicholas H, 2004. "Decision Making in Information-Rich Environments: The Role of Information Structure," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 30(4), pages 473-486, March.
    4. Moser, Kilian J. & Tumasjan, Andranik & Welpe, Isabell M., 2017. "Small but attractive: Dimensions of new venture employer attractiveness and the moderating role of applicants' entrepreneurial behaviors," Journal of Business Venturing, Elsevier, vol. 32(5), pages 588-610.
    5. Louis Guttman, 1945. "A basis for analyzing test-retest reliability," Psychometrika, Springer;The Psychometric Society, vol. 10(4), pages 255-282, December.
    6. Markku Maula & Wouter Stam, 2020. "Enhancing Rigor in Quantitative Entrepreneurship Research," Entrepreneurship Theory and Practice, , vol. 44(6), pages 1059-1090, November.
    7. Wennberg, Karl & Anderson, Brian S., 2020. "Editorial: Enhancing the exploration and communication of quantitative entrepreneurship research," Journal of Business Venturing, Elsevier, vol. 35(3).
    8. David Reibstein & John E. G. Bateson & William Boulding, 1988. "Conjoint Analysis Reliability: Empirical Findings," Marketing Science, INFORMS, vol. 7(3), pages 271-286.
    9. David A. Hensher, 2006. "How do respondents process stated choice experiments? Attribute consideration under varying information load," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 21(6), pages 861-878.
    10. Green, Paul E & Srinivasan, V, 1978. "Conjoint Analysis in Consumer Research: Issues and Outlook," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 5(2), pages 103-123, Se.
    11. Grégoire, Denis A. & Binder, Julia K. & Rauch, Andreas, 2019. "Navigating the validity tradeoffs of entrepreneurship research experiments: A systematic review and best-practice suggestions," Journal of Business Venturing, Elsevier, vol. 34(2), pages 284-310.
    12. Busenitz, Lowell W. & Barney, Jay B., 1997. "Differences between entrepreneurs and managers in large organizations: Biases and heuristics in strategic decision-making," Journal of Business Venturing, Elsevier, vol. 12(1), pages 9-30, January.
    13. Brian S. Anderson & Jens Schueler & Matthias Baum & William J. Wales & Vishal K. Gupta, 2022. "The Chicken or the Egg? Causal Inference in Entrepreneurial Orientation–Performance Research," Entrepreneurship Theory and Practice, , vol. 46(6), pages 1569-1596, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sumaya Islam & Tobias Buchta & Colin Wooldridge, 2026. "Can You Have It All? Trade-Offs, Tensions, And The Esg Tightrope In Entrepreneurial Decision-Making," Working Papers Dissertations 169, Paderborn University, Faculty of Business Administration and Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Newbert, Scott L. & Kher, Romi & Yang, Shu, 2022. "Now that's interesting and important! Moving beyond averages to increase the inferential value of empirical findings in entrepreneurship research," Journal of Business Venturing, Elsevier, vol. 37(2).
    2. Küsshauer, Alexander & Baum, Matthias, 2023. "The good, the bad and the uncertain: Employers' perceptions of former entrepreneurs," Journal of Business Venturing, Elsevier, vol. 38(2).
    3. Philipp Ego & Ulrich Pidun, 2025. "Roles of Market Characteristics and Experience in Ventures’ International Growth Mode Choice: A Conjoint Experiment," Management International Review, Springer, vol. 65(2), pages 329-357, April.
    4. Kaushik Gala & Andreas Schwab, 2026. "A Methodological Guide for Quantitative Analysis of Star Performance in Entrepreneurship," Entrepreneurship Theory and Practice, , vol. 50(3), pages 931-973, May.
    5. Teichert, Thorsten Andreas, 1997. "Schätzgenauigkeit von Conjoint-Analysen," Manuskripte aus den Instituten für Betriebswirtschaftslehre der Universität Kiel 444, Christian-Albrechts-Universität zu Kiel, Institut für Betriebswirtschaftslehre.
    6. Sylvia Hubner & Fabian Most & Jochen Wirtz & Christine Auer, 2022. "Narratives in entrepreneurial ecosystems: drivers of effectuation versus causation," Small Business Economics, Springer, vol. 59(1), pages 211-242, June.
    7. Fraser, Iain & Balcombe, Kelvin & Williams, Louis & McSorley, Eugene, 2021. "Preference stability in discrete choice experiments. Some evidence using eye-tracking," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 94(C).
    8. Seung Hoon D. Chung & Simon C. Parker, 2023. "Founder affiliations: jobseeker reactions and impact on employee recruitment by start-up ventures," Small Business Economics, Springer, vol. 61(1), pages 259-283, June.
    9. Olivier Toubia & Duncan I. Simester & John R. Hauser & Ely Dahan, 2003. "Fast Polyhedral Adaptive Conjoint Estimation," Marketing Science, INFORMS, vol. 22(3), pages 273-303.
    10. Brian S. Anderson & Jens Schueler & Matthias Baum & William J. Wales & Vishal K. Gupta, 2022. "The Chicken or the Egg? Causal Inference in Entrepreneurial Orientation–Performance Research," Entrepreneurship Theory and Practice, , vol. 46(6), pages 1569-1596, November.
    11. de Bont, Cees J. P. M. & Schoormans, Jan P. L., 1995. "The effects of product expertise on consumer evaluations of new-product concepts," Journal of Economic Psychology, Elsevier, vol. 16(4), pages 599-615, December.
    12. Peter Gordon Roetzel, 2019. "Information overload in the information age: a review of the literature from business administration, business psychology, and related disciplines with a bibliometric approach and framework development," Business Research, Springer;German Academic Association for Business Research, vol. 12(2), pages 479-522, December.
    13. Jiabin Zhang & Joeri Hugten & Wouter Stam, 2025. "Save for a rainy day? How regional household savings constrain entrepreneurship after a natural disaster," Small Business Economics, Springer, vol. 64(4), pages 2013-2033, April.
    14. Hirschmann, Mirko & Fisch, Christian & Farny, Steffen, 2025. "Biodiversity loss and entrepreneurship: Empirical evidence on threat perceptions among primary-sector entrepreneurs in 28 European countries," Journal of Business Venturing Insights, Elsevier, vol. 23(C).
    15. Currim, Imran S. & Mintz, Ofer & Siddarth, S., 2015. "Information Accessed or Information Available? The Impact on Consumer Preferences Inferred at a Durable Product E-commerce Website," Journal of Interactive Marketing, Elsevier, vol. 29(C), pages 11-25.
    16. Markku Maula & Wouter Stam, 2020. "Enhancing Rigor in Quantitative Entrepreneurship Research," Entrepreneurship Theory and Practice, , vol. 44(6), pages 1059-1090, November.
    17. Elhem Ben Fatma & Léo-Paul Dana & Souhir Elleuch & Ezzeddine Ben Mohamed, 2024. "Does dispositional optimism affect entrepreneurial success? Evidence from Saudi Arabia," International Entrepreneurship and Management Journal, Springer, vol. 20(2), pages 1127-1152, June.
    18. Marie Madeleine Meurer & Maksim Belitski & Christian Fisch & Roy Thurik, 2024. "What gets published and what doesn’t? Exploring optimal distinctiveness and diverse expectations in entrepreneurship articles," Small Business Economics, Springer, vol. 63(3), pages 1139-1170, October.
    19. Lucas, David S. & Park, U. David, 2023. "The nature and origins of social venture mission: An exploratory study of political ideology and moral foundations," Journal of Business Venturing, Elsevier, vol. 38(2).
    20. Hendrik Terbeck & Verena Rieger & Niels Van Quaquebeke & Andreas Engelen, 2022. "Once a Founder, Always a Founder? The Role of External Former Founders in Corporate Boards," Journal of Management Studies, Wiley Blackwell, vol. 59(5), pages 1284-1314, July.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:entthe:v:48:y:2024:i:2:p:742-757. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.