IDEAS home Printed from https://ideas.repec.org/a/wly/syseng/v23y2020i4p395-408.html
   My bibliography  Save this article

Technology readiness levels: Shortcomings and improvement opportunities

Author

Listed:
  • Alison L. Olechowski
  • Steven D. Eppinger
  • Nitin Joglekar
  • Katharina Tomaschek

Abstract

The technology readiness level (TRL) scale was developed at the National Aeronautics and Space Administration (NASA) in the 1970s as a standardized technology maturity assessment tool for use in complex system development. Today, TRL assessments are used to make multimillion‐dollar decisions at NASA and beyond, yet anecdotal evidence suggests that there are challenges associated with TRL use in practice. In this paper, we systematically uncover the practitioners' view, first via 19 interviews with employees from seven organizations. We identify 15 challenges of TRL implementations in three categories: system complexity, planning and review, and validity of assessment. Next, we prioritize these challenges via a survey of TRL practitioners, using a best‐worst choice experiment. Finally, we identify best practices and proposed extensions to address the challenges. We find that system complexity challenges are most critical to TRL users, despite being addressed in the literature. We posit that addressing these opportunities could result in substantial improvements to decision processes and outcomes in complex engineering projects.

Suggested Citation

  • Alison L. Olechowski & Steven D. Eppinger & Nitin Joglekar & Katharina Tomaschek, 2020. "Technology readiness levels: Shortcomings and improvement opportunities," Systems Engineering, John Wiley & Sons, vol. 23(4), pages 395-408, July.
    • Handle: RePEc:wly:syseng:v:23:y:2020:i:4:p:395-408
    DOI: 10.1002/sys.21533
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/sys.21533
    Download Restriction: no
    File URL: https://libkey.io/10.1002/sys.21533?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. V. Krishnan & Shantanu Bhattacharya, 2002. "Technology Selection and Commitment in New Product Development: The Role of Uncertainty and Design Flexibility," Management Science, INFORMS, vol. 48(3), pages 313-327, March.
    2. Iansiti, Marco, 1995. "Technology integration: Managing technological evolution in a complex environment," Research Policy, Elsevier, vol. 24(4), pages 521-542, July.
    3. Eppinger, Steven D. & Browning, Tyson R., 2012. "Design Structure Matrix Methods and Applications," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262017520, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Angelica Serna-Herrera & Oscar Mauricio Caicedo Rendón & Wilfred Rivera Martínez, 2025. "An Approach for the Development and Maturation of ICT Products," Administrative Sciences, MDPI, vol. 15(10), pages 1-28, September.
    2. Matheus Oliveira Azzi & Márcia Siqueira Rapini & Márcia Dias Diniz Costa & Hudson Luiz Silva de Miranda & Cassiano Rabelo e Silva & Ado Jório de Vasconcelos, 2024. "O financiamento a empreendimentos de base tecnológica: o caso FABNS," Textos para Discussão Cedeplar-UFMG 670, Cedeplar, Universidade Federal de Minas Gerais.
    3. Nitin Joglekar & Edward G. Anderson & Kyungmin (Brad) Lee & Geoffrey Parker & Ettore Settanni & Jagjit Singh Srai, 2022. "Configuration of digital and physical infrastructure platforms: Private and public perspectives," Production and Operations Management, Production and Operations Management Society, vol. 31(12), pages 4515-4528, December.
    4. Svetlana Klessova & Sebastian Engell & Catherine Thomas, 2023. "The interplay between the contextual conditions and the advancement of the technological maturity in inter‐organisational collaborative R&D projects: a qualitative study," Post-Print halshs-04314148, HAL.
    5. Ferreira, Paula & Rocha, Ana & Araujo, Madalena & Afonso, Joao L. & Antunes, Carlos Henggeler & Lopes, Marta A.R. & Osório, Gerardo J. & Catalão, João P.S. & Lopes, João Peças, 2023. "Assessing the societal impact of smart grids: Outcomes of a collaborative research project," Technology in Society, Elsevier, vol. 72(C).
    6. Klessova, Svetlana & Engell, Sebastian & Thomas, Catherine, 2022. "Assessment of the advancement of market-upstream innovations and of the performance of research and innovation projects," Technovation, Elsevier, vol. 116(C).
    7. Zdeněk Jegla & Silvio Bonaita & Komi Apélété Amou & Marcus Reppich, 2025. "Assessing the Theoretical Biohydrogen Potential from Agricultural Residues Using Togo as an Example," Energies, MDPI, vol. 18(17), pages 1-27, September.
    8. Anaïs Garin & Mathias Béjean & Yasmine Saleh & Robert Picard & Thomas Lihoreau, 2024. "A maturity model to align innovation ecosystem actors in health: The case of the Concept Maturity Levels [Une métrique de maturité pour aligner les acteurs d’écosystèmes d’innovation en santé : Le ," Post-Print hal-05014162, HAL.
    9. Svetlana Klessova & Sebastian Engell & Catherine Thomas, 2022. "Dynamics of couplings and their implications in inter-organizational multi-actor research and innovation projects," Post-Print hal-03690108, HAL.
    10. Svetlana Klessova & Sebastian Engell & Catherine Thomas, 2022. "Assessment of the advancement of market-upstream innovations and of the performance of research and innovation projects," Post-Print hal-03636260, HAL.
    11. Kostas Selviaridis & Martin Spring, 2022. "Fostering SME supplier‐enabled innovation in the supply chain: The role of innovation policy," Journal of Supply Chain Management, Institute for Supply Management, vol. 58(1), pages 92-123, January.
    12. Mueller, Christoph E., 2023. "Startup grants and the development of academic startup projects during funding: Quasi-experimental evidence from the German ‘EXIST – Business startup grant’," Journal of Business Venturing Insights, Elsevier, vol. 20(C).
    13. Eric Tremolada Álvarez (Editor), 2022. "Desarrollo sostenible e integración," Books, Universidad Externado de Colombia, Facultad de Derecho, number 1354, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Magistretti, Stefano & Dell'Era, Claudio & Verganti, Roberto, 2020. "Searching for the right application: A technology development review and research agenda," Technological Forecasting and Social Change, Elsevier, vol. 151(C).
    2. Kang Hyunsung D, 2018. "A Start-Up’s R&D Stages and the Evolution of Financing Sources: Evidence from the Biotechnology Industry," Entrepreneurship Research Journal, De Gruyter, vol. 8(3), pages 1-19, July.
    3. Sreekumar R. Bhaskaran & V. Krishnan, 2009. "Effort, Revenue, and Cost Sharing Mechanisms for Collaborative New Product Development," Management Science, INFORMS, vol. 55(7), pages 1152-1169, July.
    4. Uwe Beyer & Oliver Ullrich, 2022. "Organizational Complexity as a Contributing Factor to Underperformance," Businesses, MDPI, vol. 2(1), pages 1-15, March.
    5. Shigeno, Hidenori & Matsuzaki, Taisuke & Ueki, Yasushi & Tsuji, Masatsugu, 2023. "The Effect of the Covid-19 Pandemic on the Innovation Process of Small and Medium-sized Regional Firms," 32nd European Regional ITS Conference, Madrid 2023: Realising the digital decade in the European Union – Easier said than done? 278018, International Telecommunications Society (ITS).
    6. Lee, Changyong & Cho, Yangrae & Seol, Hyeonju & Park, Yongtae, 2012. "A stochastic patent citation analysis approach to assessing future technological impacts," Technological Forecasting and Social Change, Elsevier, vol. 79(1), pages 16-29.
    7. Morgan Dwyer & Bruce Cameron & Zoe Szajnfarber, 2015. "A Framework for Studying Cost Growth on Complex Acquisition Programs," Systems Engineering, John Wiley & Sons, vol. 18(6), pages 568-583, November.
    8. Mohammad Hossein Azin & Hessam Zandhessami, 2025. "Strategic Alignment Patterns in National AI Policies," Papers 2507.05400, arXiv.org, revised Jul 2025.
    9. Robert Schmidt & Kasper Sanchez Vibaek & Simon Austin, 2014. "Evaluating the adaptability of an industrialized building using dependency structure matrices," Construction Management and Economics, Taylor & Francis Journals, vol. 32(1-2), pages 160-182, February.
    10. Kaushik Sinha & Seok‐Youn Han & Eun Suk Suh, 2020. "Design structure matrix‐based modularization approach for complex systems with multiple design constraints," Systems Engineering, John Wiley & Sons, vol. 23(2), pages 211-220, March.
    11. Burcu Tan & Edward G. Anderson, Jr. & Geoffrey G. Parker, 2020. "Platform Pricing and Investment to Drive Third-Party Value Creation in Two-Sided Networks," Information Systems Research, INFORMS, vol. 31(1), pages 217-239, March.
    12. Amirmahmood Amini Sedeh & Rosa Caiazza & Amir Pezeshkan, 2023. "Unraveling the resource puzzle: exploring entrepreneurial resource management and the quest for new venture success," The Journal of Technology Transfer, Springer, vol. 48(5), pages 1552-1573, October.
    13. Juntao Zhang & Cecilia Haskins & Yiliu Liu & Mary Ann Lundteigen, 2018. "A systems engineering–based approach for framing reliability, availability, and maintainability: A case study for subsea design," Systems Engineering, John Wiley & Sons, vol. 21(6), pages 576-592, November.
    14. Nazarizadeh, Farzaneh & Alemtabriz, Akbar & Zandieh, Mostafa & Raad, Abbas, 2022. "An analytical model for reliability assessment of the rail system considering dependent failures (case study of Iranian railway)," Reliability Engineering and System Safety, Elsevier, vol. 227(C).
    15. Khraisha, Tamer, 2020. "Complex economic problems and fitness landscapes: Assessment and methodological perspectives," Structural Change and Economic Dynamics, Elsevier, vol. 52(C), pages 390-407.
    16. Andreas Poller, 2020. "Exploring and managing the complexity of large infrastructure projects with network theory and model‐based systems engineering—The example of radioactive waste disposal," Systems Engineering, John Wiley & Sons, vol. 23(4), pages 443-459, July.
    17. Lin, Jun & Qian, Yanjun & Cui, Wentian & Goh, Thong Ngee, 2015. "An effective approach for scheduling coupled activities in development projects," European Journal of Operational Research, Elsevier, vol. 243(1), pages 97-108.
    18. Annika Lorenz & Michael Raven & Knut Blind, 2019. "The role of standardization at the interface of product and process development in biotechnology," The Journal of Technology Transfer, Springer, vol. 44(4), pages 1097-1133, August.
    19. Sam Ransbotham & Sabyasachi Mitra, 2010. "Target Age and the Acquisition of Innovation in High-Technology Industries," Management Science, INFORMS, vol. 56(11), pages 2076-2093, November.
    20. Christopher M. Schlick & Soenke Duckwitz & Sebastian Schneider, 2013. "Project dynamics and emergent complexity," Computational and Mathematical Organization Theory, Springer, vol. 19(4), pages 480-515, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:syseng:v:23:y:2020:i:4:p:395-408. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1002/(ISSN)1520-6858 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.