IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v194y2020ics0951832017310293.html
   My bibliography  Save this article

Expert judgments for performance shaping Factors’ multiplier design in human reliability analysis

Author

Listed:
  • Liu, Peng
  • Qiu, Yongping
  • Hu, Juntao
  • Tong, Jiejuan
  • Zhao, Jun
  • Li, Zhizhong

Abstract

Human reliability analysis (HRA) still heavily relies on expert judgments to generate reliability data. There exists a widely recognized need to validate and justify the reliability data obtained from expert judgments. For demonstrating such effort, we provide a template of how we base expert elicitations and empirical studies to derive the multipliers of performance shaping factors (PSFs). We applied two expert judgment techniques—absolute probability judgment (APJ) and ratio magnitude estimation (RME)—to update the PSF multiplier design in Standardized Plant Analysis of Risk-Human Reliability Analysis (SPAR-H). Licensed operators (N = 17) from a nuclear power plant were recruited. It is found that APJ and RME have acceptable inter-rater reliability and convergent validity between them. The multipliers estimated by APJ and RME were compared with those from empirical studies in the human performance literature. Certain consistencies between these heterogeneous data sources were found. Combining these heterogeneous data, we suggested the multiplier design of PSFs for SPAR-H. We also bridged the relationship between every PSF and its psychological mechanism to trigger human errors. Our work might suggest the appropriateness of expert elicitations in generating useful data for HRA, and strengthen the empirical and psychological foundations of PSF-based HRA methods.

Suggested Citation

  • Liu, Peng & Qiu, Yongping & Hu, Juntao & Tong, Jiejuan & Zhao, Jun & Li, Zhizhong, 2020. "Expert judgments for performance shaping Factors’ multiplier design in human reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 194(C).
    • Handle: RePEc:eee:reensy:v:194:y:2020:i:c:s0951832017310293
    DOI: 10.1016/j.ress.2018.12.022
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832017310293
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ress.2018.12.022?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. B. S. Dhillon, 2014. "Human Factors in Power Plant Maintenance," Springer Series in Reliability Engineering, in: Human Reliability, Error, and Human Factors in Power Generation, edition 127, chapter 0, pages 123-133, Springer.
    2. James Chang, Y. & Bley, Dennis & Criscione, Lawrence & Kirwan, Barry & Mosleh, Ali & Madary, Todd & Nowell, Rodney & Richards, Robert & Roth, Emilie M. & Sieben, Scott & Zoulis, Antonios, 2014. "The SACADA database for human reliability and human performance," Reliability Engineering and System Safety, Elsevier, vol. 125(C), pages 117-133.
    3. Bing Wu & Xinping Yan & Yang Wang & C. Guedes Soares, 2017. "An Evidential Reasoning‐Based CREAM to Human Reliability Analysis in Maritime Accident Process," Risk Analysis, John Wiley & Sons, vol. 37(10), pages 1936-1957, October.
    4. Kim, Yochan & Park, Jinkyun & Jung, Wondea, 2017. "A quantitative measure of fitness for duty and work processes for human reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 167(C), pages 595-601.
    5. David V. Budescu & Eva Chen, 2015. "Identifying Expertise to Extract the Wisdom of Crowds," Management Science, INFORMS, vol. 61(2), pages 267-280, February.
    6. Ekanem, Nsimah J. & Mosleh, Ali & Shen, Song-Hua, 2016. "Phoenix – A model-based Human Reliability Analysis methodology: Qualitative Analysis Procedure," Reliability Engineering and System Safety, Elsevier, vol. 145(C), pages 301-315.
    7. Griffith, Candice D. & Mahadevan, Sankaran, 2011. "Inclusion of fatigue effects in human reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 96(11), pages 1437-1447.
    8. George E. Apostolakis, 2004. "How Useful Is Quantitative Risk Assessment?," Risk Analysis, John Wiley & Sons, vol. 24(3), pages 515-520, June.
    9. B. S. Dhillon, 2014. "Human Error in Power Generation," Springer Series in Reliability Engineering, in: Human Reliability, Error, and Human Factors in Power Generation, edition 127, chapter 0, pages 93-105, Springer.
    10. Podofillini, L. & Dang, V.N., 2013. "A Bayesian approach to treat expert-elicited probabilities in human reliability analysis model construction," Reliability Engineering and System Safety, Elsevier, vol. 117(C), pages 52-64.
    11. Robert T. Clemen & Robert L. Winkler, 1999. "Combining Probability Distributions From Experts in Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 19(2), pages 187-203, April.
    12. Stephen C. Hora, 2010. "An Analytic Method for Evaluating the Performance of Aggregation Rules for Probability Densities," Operations Research, INFORMS, vol. 58(5), pages 1440-1449, October.
    13. Park, Jinkyun & Kim, Yochan & Jung, Wondea, 2018. "Calculating nominal human error probabilities from the operation experience of domestic nuclear power plants," Reliability Engineering and System Safety, Elsevier, vol. 170(C), pages 215-225.
    14. Jay R. Galbraith, 1974. "Organization Design: An Information Processing View," Interfaces, INFORMS, vol. 4(3), pages 28-36, May.
    15. Mkrtchyan, L. & Podofillini, L. & Dang, V.N., 2016. "Methods for building Conditional Probability Tables of Bayesian Belief Networks from limited judgment: An evaluation for Human Reliability Application," Reliability Engineering and System Safety, Elsevier, vol. 151(C), pages 93-112.
    16. Groth, Katrina M. & Smith, Curtis L. & Swiler, Laura P., 2014. "A Bayesian method for using simulator data to enhance human error probabilities assigned by existing HRA methods," Reliability Engineering and System Safety, Elsevier, vol. 128(C), pages 32-40.
    17. M.V. Ramana & Ashwin Kumar, 2014. "'One in infinity': failing to learn from accidents and implications for nuclear safety in India," Journal of Risk Research, Taylor & Francis Journals, vol. 17(1), pages 23-42, January.
    18. B. S. Dhillon, 2014. "Human Error in Power Plant Maintenance," Springer Series in Reliability Engineering, in: Human Reliability, Error, and Human Factors in Power Generation, edition 127, chapter 0, pages 135-149, Springer.
    19. Groth, Katrina M. & Swiler, Laura P., 2013. "Bridging the gap between HRA research and HRA practice: A Bayesian network version of SPAR-H," Reliability Engineering and System Safety, Elsevier, vol. 115(C), pages 33-42.
    20. Kim, Yochan & Park, Jinkyun & Jung, Wondea & Choi, Sun Yeong & Kim, Seunghwan, 2018. "Estimating the quantitative relation between PSFs and HEPs from full-scope simulator data," Reliability Engineering and System Safety, Elsevier, vol. 173(C), pages 12-22.
    21. Rajagopal, 2014. "The Human Factors," Palgrave Macmillan Books, in: Architecting Enterprise, chapter 9, pages 225-249, Palgrave Macmillan.
    22. De Ambroggi, Massimiliano & Trucco, Paolo, 2011. "Modelling and assessment of dependent performance shaping factors through Analytic Network Process," Reliability Engineering and System Safety, Elsevier, vol. 96(7), pages 849-860.
    23. Griffith, Candice D. & Mahadevan, Sankaran, 2015. "Human reliability under sleep deprivation: Derivation of performance shaping factor multipliers from empirical data," Reliability Engineering and System Safety, Elsevier, vol. 144(C), pages 23-34.
    24. Kim, Yochan & Park, Jinkyun & Jung, Wondea & Jang, Inseok & Hyun Seong, Poong, 2015. "A statistical approach to estimating effects of performance shaping factors on human error probabilities of soft controls," Reliability Engineering and System Safety, Elsevier, vol. 142(C), pages 378-387.
    25. B. S. Dhillon, 2014. "Human Factors in Power Generation," Springer Series in Reliability Engineering, in: Human Reliability, Error, and Human Factors in Power Generation, edition 127, chapter 0, pages 81-92, Springer.
    26. B. S. Dhillon, 2014. "Human Reliability, Error, and Human Factors in Power Generation," Springer Series in Reliability Engineering, Springer, edition 127, number 978-3-319-04019-6, January.
    27. Laumann, Karin & Rasmussen, Martin, 2016. "Suggested improvements to the definitions of Standardized Plant Analysis of Risk-Human Reliability Analysis (SPAR-H) performance shaping factors, their levels and multipliers and the nominal tasks," Reliability Engineering and System Safety, Elsevier, vol. 145(C), pages 287-300.
    28. Zwirglmaier, Kilian & Straub, Daniel & Groth, Katrina M., 2017. "Capturing cognitive causal paths in human reliability analysis with Bayesian network models," Reliability Engineering and System Safety, Elsevier, vol. 158(C), pages 117-129.
    29. Peng Liu & Zhizhong Li, 2014. "Human Error Data Collection and Comparison with Predictions by SPAR‐H," Risk Analysis, John Wiley & Sons, vol. 34(9), pages 1706-1719, September.
    30. Liu, Peng & Lyu, Xi & Qiu, Yongping & He, Jiandong & Tong, Jiejuan & Zhao, Jun & Li, Zhizhong, 2017. "Identifying key performance shaping factors in digital main control rooms of nuclear power plants: A risk-based approach," Reliability Engineering and System Safety, Elsevier, vol. 167(C), pages 264-275.
    31. Xu, Song & Song, Fei & Li, Zhizhong & Zhao, Qianyi & Luo, Wei & He, Xuhong & Salvendy, Gavriel, 2008. "An ergonomics study of computerized emergency operating procedures: Presentation style, task complexity, and training level," Reliability Engineering and System Safety, Elsevier, vol. 93(10), pages 1500-1511.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhao, Yunfei & Smidts, Carol, 2021. "CMS-BN: A cognitive modeling and simulation environment for human performance assessment, part 1 — methodology," Reliability Engineering and System Safety, Elsevier, vol. 213(C).
    2. Catelani, Marcantonio & Ciani, Lorenzo & Guidi, Giulia & Patrizi, Gabriele, 2021. "An enhanced SHERPA (E-SHERPA) method for human reliability analysis in railway engineering," Reliability Engineering and System Safety, Elsevier, vol. 215(C).
    3. Liu, Jianqiao & Zou, Yanhua & Wang, Wei & Zhang, Li & Liu, Xueyang & Ding, Qianqiao & Qin, Zhuomin & ÄŒepin, Marko, 2021. "Analysis of dependencies among performance shaping factors in human reliability analysis based on a system dynamics approach," Reliability Engineering and System Safety, Elsevier, vol. 215(C).
    4. Zhou, Jian-Lan & Tu, Ren-Fang & Xiao, Hai, 2022. "Large-scale group decision-making to facilitate inter-rater reliability of human-factors analysis for the railway system," Reliability Engineering and System Safety, Elsevier, vol. 228(C).
    5. Zhao, Yunfei & Smidts, Carol, 2021. "CMS-BN: A cognitive modeling and simulation environment for human performance assessment, part 2 — Application," Reliability Engineering and System Safety, Elsevier, vol. 213(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Patriarca, Riccardo & Ramos, Marilia & Paltrinieri, Nicola & Massaiu, Salvatore & Costantino, Francesco & Di Gravio, Giulio & Boring, Ronald Laurids, 2020. "Human reliability analysis: Exploring the intellectual structure of a research field," Reliability Engineering and System Safety, Elsevier, vol. 203(C).
    2. Liu, Peng & Lyu, Xi & Qiu, Yongping & He, Jiandong & Tong, Jiejuan & Zhao, Jun & Li, Zhizhong, 2017. "Identifying key performance shaping factors in digital main control rooms of nuclear power plants: A risk-based approach," Reliability Engineering and System Safety, Elsevier, vol. 167(C), pages 264-275.
    3. Paglioni, Vincent P. & Groth, Katrina M., 2022. "Dependency definitions for quantitative human reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 220(C).
    4. Liu, Jianqiao & Zou, Yanhua & Wang, Wei & Zio, Enrico & Yuan, Chengwei & Wang, Taorui & Jiang, Jianjun, 2022. "A Bayesian belief network framework for nuclear power plant human reliability analysis accounting for dependencies among performance shaping factors," Reliability Engineering and System Safety, Elsevier, vol. 228(C).
    5. Zarei, Esmaeil & Khan, Faisal & Abbassi, Rouzbeh, 2021. "Importance of human reliability in process operation: A critical analysis," Reliability Engineering and System Safety, Elsevier, vol. 211(C).
    6. Morais, Caroline & Estrada-Lugo, Hector Diego & Tolo, Silvia & Jacques, Tiago & Moura, Raphael & Beer, Michael & Patelli, Edoardo, 2022. "Robust data-driven human reliability analysis using credal networks," Reliability Engineering and System Safety, Elsevier, vol. 218(PA).
    7. Abrishami, Shokoufeh & Khakzad, Nima & Hosseini, Seyed Mahmoud, 2020. "A data-based comparison of BN-HRA models in assessing human error probability: An offshore evacuation case study," Reliability Engineering and System Safety, Elsevier, vol. 202(C).
    8. Groth, Katrina M. & Smith, Reuel & Moradi, Ramin, 2019. "A hybrid algorithm for developing third generation HRA methods using simulator data, causal models, and cognitive science," Reliability Engineering and System Safety, Elsevier, vol. 191(C).
    9. Greco, Salvatore F. & Podofillini, Luca & Dang, Vinh N., 2021. "A Bayesian model to treat within-category and crew-to-crew variability in simulator data for Human Reliability Analysis," Reliability Engineering and System Safety, Elsevier, vol. 206(C).
    10. Shirley, Rachel Benish & Smidts, Carol & Zhao, Yunfei, 2020. "Development of a quantitative Bayesian network mapping objective factors to subjective performance shaping factor evaluations: An example using student operators in a digital nuclear power plant simul," Reliability Engineering and System Safety, Elsevier, vol. 194(C).
    11. Kim, Yochan & Park, Jinkyun & Jung, Wondea & Jang, Inseok & Hyun Seong, Poong, 2015. "A statistical approach to estimating effects of performance shaping factors on human error probabilities of soft controls," Reliability Engineering and System Safety, Elsevier, vol. 142(C), pages 378-387.
    12. Podofillini, Luca & Reer, Bernhard & Dang, Vinh N., 2023. "A traceable process to develop Bayesian networks from scarce data and expert judgment: A human reliability analysis application," Reliability Engineering and System Safety, Elsevier, vol. 230(C).
    13. Kim, Yochan & Park, Jinkyun, 2019. "Incorporating prior knowledge with simulation data to estimate PSF multipliers using Bayesian logistic regression," Reliability Engineering and System Safety, Elsevier, vol. 189(C), pages 210-217.
    14. Wang, Wei & Di Maio, Francesco & Zio, Enrico, 2020. "Considering the human operator cognitive process for the interpretation of diagnostic outcomes related to component failures and cyber security attacks," Reliability Engineering and System Safety, Elsevier, vol. 202(C).
    15. Wu, Bing & Yip, Tsz Leung & Yan, Xinping & Guedes Soares, C., 2022. "Review of techniques and challenges of human and organizational factors analysis in maritime transportation," Reliability Engineering and System Safety, Elsevier, vol. 219(C).
    16. Kim, Yochan & Park, Jinkyun & Jung, Wondea, 2017. "A quantitative measure of fitness for duty and work processes for human reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 167(C), pages 595-601.
    17. Peng Liu & Zhizhong Li, 2014. "Human Error Data Collection and Comparison with Predictions by SPAR‐H," Risk Analysis, John Wiley & Sons, vol. 34(9), pages 1706-1719, September.
    18. Li, Jue & Li, Heng & Wang, Fan & Cheng, Andy S.K. & Yang, Xincong & Wang, Hongwei, 2021. "Proactive analysis of construction equipment operators’ hazard perception error based on cognitive modeling and a dynamic Bayesian network," Reliability Engineering and System Safety, Elsevier, vol. 205(C).
    19. Liu, Jianqiao & Zou, Yanhua & Wang, Wei & Zhang, Li & Liu, Xueyang & Ding, Qianqiao & Qin, Zhuomin & ÄŒepin, Marko, 2021. "Analysis of dependencies among performance shaping factors in human reliability analysis based on a system dynamics approach," Reliability Engineering and System Safety, Elsevier, vol. 215(C).
    20. Zhao, Yunfei & Smidts, Carol, 2021. "CMS-BN: A cognitive modeling and simulation environment for human performance assessment, part 1 — methodology," Reliability Engineering and System Safety, Elsevier, vol. 213(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:194:y:2020:i:c:s0951832017310293. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.