IDEAS home Printed from https://ideas.repec.org/a/wly/syseng/v23y2020i4p395-408.html
   My bibliography  Save this article

Technology readiness levels: Shortcomings and improvement opportunities

Author

Listed:
  • Alison L. Olechowski
  • Steven D. Eppinger
  • Nitin Joglekar
  • Katharina Tomaschek

Abstract

The technology readiness level (TRL) scale was developed at the National Aeronautics and Space Administration (NASA) in the 1970s as a standardized technology maturity assessment tool for use in complex system development. Today, TRL assessments are used to make multimillion‐dollar decisions at NASA and beyond, yet anecdotal evidence suggests that there are challenges associated with TRL use in practice. In this paper, we systematically uncover the practitioners' view, first via 19 interviews with employees from seven organizations. We identify 15 challenges of TRL implementations in three categories: system complexity, planning and review, and validity of assessment. Next, we prioritize these challenges via a survey of TRL practitioners, using a best‐worst choice experiment. Finally, we identify best practices and proposed extensions to address the challenges. We find that system complexity challenges are most critical to TRL users, despite being addressed in the literature. We posit that addressing these opportunities could result in substantial improvements to decision processes and outcomes in complex engineering projects.

Suggested Citation

  • Alison L. Olechowski & Steven D. Eppinger & Nitin Joglekar & Katharina Tomaschek, 2020. "Technology readiness levels: Shortcomings and improvement opportunities," Systems Engineering, John Wiley & Sons, vol. 23(4), pages 395-408, July.
    • Handle: RePEc:wly:syseng:v:23:y:2020:i:4:p:395-408
    DOI: 10.1002/sys.21533
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/sys.21533
    Download Restriction: no
    File URL: https://libkey.io/10.1002/sys.21533?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. V. Krishnan & Shantanu Bhattacharya, 2002. "Technology Selection and Commitment in New Product Development: The Role of Uncertainty and Design Flexibility," Management Science, INFORMS, vol. 48(3), pages 313-327, March.
    2. Iansiti, Marco, 1995. "Technology integration: Managing technological evolution in a complex environment," Research Policy, Elsevier, vol. 24(4), pages 521-542, July.
    3. Eppinger, Steven D. & Browning, Tyson R., 2012. "Design Structure Matrix Methods and Applications," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262017520, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nitin Joglekar & Edward G. Anderson & Kyungmin (Brad) Lee & Geoffrey Parker & Ettore Settanni & Jagjit Singh Srai, 2022. "Configuration of digital and physical infrastructure platforms: Private and public perspectives," Production and Operations Management, Production and Operations Management Society, vol. 31(12), pages 4515-4528, December.
    2. Ferreira, Paula & Rocha, Ana & Araujo, Madalena & Afonso, Joao L. & Antunes, Carlos Henggeler & Lopes, Marta A.R. & Osório, Gerardo J. & Catalão, João P.S. & Lopes, João Peças, 2023. "Assessing the societal impact of smart grids: Outcomes of a collaborative research project," Technology in Society, Elsevier, vol. 72(C).
    3. Klessova, Svetlana & Engell, Sebastian & Thomas, Catherine, 2022. "Assessment of the advancement of market-upstream innovations and of the performance of research and innovation projects," Technovation, Elsevier, vol. 116(C).
    4. Svetlana Klessova & Sebastian Engell & Catherine Thomas, 2022. "Dynamics of couplings and their implications in inter-organizational multi-actor research and innovation projects," Post-Print hal-03690108, HAL.
    5. Svetlana Klessova & Sebastian Engell & Catherine Thomas, 2022. "Assessment of the advancement of market-upstream innovations and of the performance of research and innovation projects," Post-Print hal-03636260, HAL.
    6. Kostas Selviaridis & Martin Spring, 2022. "Fostering SME supplier‐enabled innovation in the supply chain: The role of innovation policy," Journal of Supply Chain Management, Institute for Supply Management, vol. 58(1), pages 92-123, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Magistretti, Stefano & Dell'Era, Claudio & Verganti, Roberto, 2020. "Searching for the right application: A technology development review and research agenda," Technological Forecasting and Social Change, Elsevier, vol. 151(C).
    2. Sreekumar R. Bhaskaran & V. Krishnan, 2009. "Effort, Revenue, and Cost Sharing Mechanisms for Collaborative New Product Development," Management Science, INFORMS, vol. 55(7), pages 1152-1169, July.
    3. Kang Hyunsung D, 2018. "A Start-Up’s R&D Stages and the Evolution of Financing Sources: Evidence from the Biotechnology Industry," Entrepreneurship Research Journal, De Gruyter, vol. 8(3), pages 1-19, July.
    4. Lorenzo Ciapetti, 2011. "Technological Change, Knowledge Integration and Adaptive Processes: The Mechatronic Evolution of the Reggio Emilia District," Chapters, in: Paul L. Robertson & David Jacobson (ed.), Knowledge Transfer and Technology Diffusion, chapter 5, Edward Elgar Publishing.
    5. Uwe Beyer & Oliver Ullrich, 2022. "Organizational Complexity as a Contributing Factor to Underperformance," Businesses, MDPI, vol. 2(1), pages 1-15, March.
    6. Shigeno, Hidenori & Matsuzaki, Taisuke & Ueki, Yasushi & Tsuji, Masatsugu, 2023. "The Effect of the Covid-19 Pandemic on the Innovation Process of Small and Medium-sized Regional Firms," 32nd European Regional ITS Conference, Madrid 2023: Realising the digital decade in the European Union – Easier said than done? 278018, International Telecommunications Society (ITS).
    7. Lee, Changyong & Cho, Yangrae & Seol, Hyeonju & Park, Yongtae, 2012. "A stochastic patent citation analysis approach to assessing future technological impacts," Technological Forecasting and Social Change, Elsevier, vol. 79(1), pages 16-29.
    8. Morgan Dwyer & Bruce Cameron & Zoe Szajnfarber, 2015. "A Framework for Studying Cost Growth on Complex Acquisition Programs," Systems Engineering, John Wiley & Sons, vol. 18(6), pages 568-583, November.
    9. Félicia Saïah & Diego Vega & Harwin de Vries & Joakim Kembro, 2023. "Process modularity, supply chain responsiveness, and moderators: The Médecins Sans Frontières response to the Covid‐19 pandemic," Production and Operations Management, Production and Operations Management Society, vol. 32(5), pages 1490-1511, May.
    10. Axarloglou, Kostas & Visvikis, Ilias & Zarkos, Stefanos, 2013. "The time dimension and value of flexibility in resource allocation: The case of the maritime industry," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 52(C), pages 35-48.
    11. Junguang Zhang & Xiwei Song & Hongyu Chen & Ruixia (Sandy) Shi, 2016. "Determination of critical chain project buffer based on information flow interactions," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 67(9), pages 1146-1157, September.
    12. Aditya Vedantam & Ananth Iyer, 2021. "Capacity Investment under Bayesian Information Updates at Reporting Periods: Model and Application," Production and Operations Management, Production and Operations Management Society, vol. 30(8), pages 2707-2725, August.
    13. Liu, Zhixue & Ding, Ronggui & Wang, Lei & Song, Rui & Song, Xinyi, 2023. "Cooperation in an uncertain environment: The impact of stakeholders' concerted action on collaborative innovation projects risk management," Technological Forecasting and Social Change, Elsevier, vol. 196(C).
    14. Robert Schmidt & Kasper Sanchez Vibaek & Simon Austin, 2014. "Evaluating the adaptability of an industrialized building using dependency structure matrices," Construction Management and Economics, Taylor & Francis Journals, vol. 32(1-2), pages 160-182, February.
    15. Lim, Wei Shi & Tang, Christopher S., 2006. "Optimal product rollover strategies," European Journal of Operational Research, Elsevier, vol. 174(2), pages 905-922, October.
    16. Subarna Basnet & Christopher L Magee, 2017. "Artifact interactions retard technological improvement: An empirical study," PLOS ONE, Public Library of Science, vol. 12(8), pages 1-17, August.
    17. Kaushik Sinha & Seok‐Youn Han & Eun Suk Suh, 2020. "Design structure matrix‐based modularization approach for complex systems with multiple design constraints," Systems Engineering, John Wiley & Sons, vol. 23(2), pages 211-220, March.
    18. Clayton M. Christensen & Fernando F. Suárez & James M. Utterback, 1998. "Strategies for Survival in Fast-Changing Industries," Management Science, INFORMS, vol. 44(12-Part-2), pages 207-220, December.
    19. David A. Broniatowski, 2018. "Building the tower without climbing it: Progress in engineering systems," Systems Engineering, John Wiley & Sons, vol. 21(3), pages 259-281, May.
    20. Burcu Tan & Edward G. Anderson, Jr. & Geoffrey G. Parker, 2020. "Platform Pricing and Investment to Drive Third-Party Value Creation in Two-Sided Networks," Information Systems Research, INFORMS, vol. 31(1), pages 217-239, March.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:syseng:v:23:y:2020:i:4:p:395-408. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1002/(ISSN)1520-6858 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.