IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v291y2021i2p794-806.html
   My bibliography  Save this article

Combining multiple criteria analysis, mathematical programming and Monte Carlo simulation to tackle uncertainty in Research and Development project portfolio selection: A case study from Greece

Author

Listed:
  • Mavrotas, George
  • Makryvelios, Evangelos

Abstract

Research and Development (R&D) is a substantial activity in the modern economy of knowledge and therefore funding R&D activities is a challenging task for private and public institutions. The difficulty in evaluation of R&D projects is mainly the inherent uncertainty that has to be dealt with. In addition, when we have funding programs, various policy constraints for the allocation of funds must be also taken into account. In this paper we propose an R&D project portfolio selection method that deals with the inherent uncertainty of R&D project evaluation. Using decision rounds with multicriteria analysis, mathematical programming and Monte Carlo simulation in the framework of the Iterative Trichotomic Approach (ITA), we manage to tackle large problems. The method is applied in a case study from Greece with 2437 project proposals from the funding action “Research-Create-Innovate” with European and National resources. Groups of experts evaluate the projects in three criteria and complex policy constraints (geographical, sectoral etc.) are also applied. The iterative nature of ITA allows for gradually converging to the final portfolio of R&D projects. The results provide information not only about the acceptance or not of an R&D project to the final portfolio, but also deal with the main source of uncertainty which is the experts’ evaluation, providing a degree of certainty for the selected and rejected projects.

Suggested Citation

  • Mavrotas, George & Makryvelios, Evangelos, 2021. "Combining multiple criteria analysis, mathematical programming and Monte Carlo simulation to tackle uncertainty in Research and Development project portfolio selection: A case study from Greece," European Journal of Operational Research, Elsevier, vol. 291(2), pages 794-806.
    • Handle: RePEc:eee:ejores:v:291:y:2021:i:2:p:794-806
    DOI: 10.1016/j.ejor.2020.09.051
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221720308559
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2020.09.051?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Nikolaos Argyris & José Rui Figueira & Alec Morton, 2011. "Interactive Multicriteria Methods in Portfolio Decision Analysis," International Series in Operations Research & Management Science, in: Ahti Salo & Jeffrey Keisler & Alec Morton (ed.), Portfolio Decision Analysis, chapter 0, pages 107-130, Springer.
    2. Ahti Salo & Jeffrey Keisler & Alec Morton, 2011. "An Invitation to Portfolio Decision Analysis," International Series in Operations Research & Management Science, in: Ahti Salo & Jeffrey Keisler & Alec Morton (ed.), Portfolio Decision Analysis, chapter 0, pages 3-27, Springer.
    3. Mavrotas, George & Diakoulaki, Danae & Kourentzis, Athanasios, 2008. "Selection among ranked projects under segmentation, policy and logical constraints," European Journal of Operational Research, Elsevier, vol. 187(1), pages 177-192, May.
    4. Oral, Muhittin & Kettani, Ossama & Cinar, Unver, 2001. "Project evaluation and selection in a network of collaboration: A consensual disaggregation multi-criterion approach," European Journal of Operational Research, Elsevier, vol. 130(2), pages 332-346, April.
    5. Kamal Golabi & Craig W. Kirkwood & Alan Sicherman, 1981. "Selecting a Portfolio of Solar Energy Projects Using Multiattribute Preference Theory," Management Science, INFORMS, vol. 27(2), pages 174-189, February.
    6. Wang, Juite & Hwang, W.-L., 2007. "A fuzzy set approach for R&D portfolio selection using a real options valuation model," Omega, Elsevier, vol. 35(3), pages 247-257, June.
    7. Liesiö, Juuso & Mild, Pekka & Salo, Ahti, 2008. "Robust portfolio modeling with incomplete cost information and project interdependencies," European Journal of Operational Research, Elsevier, vol. 190(3), pages 679-695, November.
    8. Alec Morton & Jeffrey M. Keisler & Ahti Salo, 2016. "Multicriteria Portfolio Decision Analysis for Project Selection," International Series in Operations Research & Management Science, in: Salvatore Greco & Matthias Ehrgott & José Rui Figueira (ed.), Multiple Criteria Decision Analysis, edition 2, chapter 0, pages 1269-1298, Springer.
    9. Mara Airoldi & Alec Morton, 2011. "Portfolio Decision Analysis for Population Health," International Series in Operations Research & Management Science, in: Ahti Salo & Jeffrey Keisler & Alec Morton (ed.), Portfolio Decision Analysis, chapter 0, pages 359-381, Springer.
    10. George Mavrotas & Olena Pechak, 2013. "The trichotomic approach for dealing with uncertainty in project portfolio selection: combining MCDA, mathematical programming and Monte Carlo simulation," International Journal of Multicriteria Decision Making, Inderscience Enterprises Ltd, vol. 3(1), pages 79-96.
    11. Klette, Tor Jakob & Moen, Jarle & Griliches, Zvi, 2000. "Do subsidies to commercial R&D reduce market failures? Microeconometric evaluation studies1," Research Policy, Elsevier, vol. 29(4-5), pages 471-495, April.
    12. Mavrotas, G. & Diakoulaki, D. & Caloghirou, Y., 2006. "Project prioritization under policy restrictions. A combination of MCDA with 0-1 programming," European Journal of Operational Research, Elsevier, vol. 171(1), pages 296-308, May.
    13. Jaffe, Adam B, 1989. "Real Effects of Academic Research," American Economic Review, American Economic Association, vol. 79(5), pages 957-970, December.
    14. Liesio, Juuso & Mild, Pekka & Salo, Ahti, 2007. "Preference programming for robust portfolio modeling and project selection," European Journal of Operational Research, Elsevier, vol. 181(3), pages 1488-1505, September.
    15. Zanakis, Stelios H. & Mandakovic, Tomislav & Gupta, Sushil K. & Sahay, Sundeep & Hong, Sungwan, 1995. "A review of program evaluation and fund allocation methods within the service and government sectors," Socio-Economic Planning Sciences, Elsevier, vol. 29(1), pages 59-79, March.
    16. ., 2019. "Stumbling into action research," Chapters, in: Convenience Triangle in White-Collar Crime, chapter 1, pages 5-17, Edward Elgar Publishing.
    17. Cranmer, Alexana & Baker, Erin & Liesiö, Juuso & Salo, Ahti, 2018. "A portfolio model for siting offshore wind farms with economic and environmental objectives," European Journal of Operational Research, Elsevier, vol. 267(1), pages 304-314.
    18. Wang, Jue & Xu, Wei & Ma, Jian & Wang, Shouyang, 2013. "A vague set based decision support approach for evaluating research funding programs," European Journal of Operational Research, Elsevier, vol. 230(3), pages 656-665.
    19. Kurth, Margaret & Keisler, Jeffrey M. & Bates, Matthew E. & Bridges, Todd S. & Summers, Jeffrey & Linkov, Igor, 2017. "A portfolio decision analysis approach to support energy research and development resource allocation," Energy Policy, Elsevier, vol. 105(C), pages 128-135.
    20. Harry Markowitz, 1952. "Portfolio Selection," Journal of Finance, American Finance Association, vol. 7(1), pages 77-91, March.
    21. Cook, Wade D. & Green, Rodney H., 2000. "Project prioritization: a resource-constrained data envelopment analysis approach," Socio-Economic Planning Sciences, Elsevier, vol. 34(2), pages 85-99, June.
    22. Eduardo Fernandez & Fernando Lopez & Jorge Navarro & Ines Vega & Igor Litvinchev, 2009. "An integrated mathematical-computer approach for R&D project selection in large public organisations," International Journal of Mathematics in Operational Research, Inderscience Enterprises Ltd, vol. 1(3), pages 372-396.
    23. Abu-Taleb, Maher F. & Mareschal, Bertrand, 1995. "Water resources planning in the Middle East: Application of the PROMETHEE V multicriteria method," European Journal of Operational Research, Elsevier, vol. 81(3), pages 500-511, March.
    24. Santamaría, Lluís & Barge-Gil, Andrés & Modrego, Aurelia, 2010. "Public selection and financing of R&D cooperative projects: Credit versus subsidy funding," Research Policy, Elsevier, vol. 39(4), pages 549-563, May.
    25. Barbara Fasolo & Alec Morton & Detlof Winterfeldt, 2011. "Behavioural Issues in Portfolio Decision Analysis," International Series in Operations Research & Management Science, in: Ahti Salo & Jeffrey Keisler & Alec Morton (ed.), Portfolio Decision Analysis, chapter 0, pages 149-165, Springer.
    26. Lieh-Ming Luo, 2012. "Optimal diversification for R&D project portfolios," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(1), pages 219-229, April.
    27. Eilat, Harel & Golany, Boaz & Shtub, Avraham, 2006. "Constructing and evaluating balanced portfolios of R&D projects with interactions: A DEA based methodology," European Journal of Operational Research, Elsevier, vol. 172(3), pages 1018-1039, August.
    28. Song, Haiyan & Qiu, Richard T.R. & Park, Jinah, 2019. "A review of research on tourism demand forecasting," Annals of Tourism Research, Elsevier, vol. 75(C), pages 338-362.
    29. Barbati, Maria & Greco, Salvatore & Kadziński, Miłosz & Słowiński, Roman, 2018. "Optimization of multiple satisfaction levels in portfolio decision analysis," Omega, Elsevier, vol. 78(C), pages 192-204.
    30. Solak, Senay & Clarke, John-Paul B. & Johnson, Ellis L. & Barnes, Earl R., 2010. "Optimization of R&D project portfolios under endogenous uncertainty," European Journal of Operational Research, Elsevier, vol. 207(1), pages 420-433, November.
    31. Lee, Hakyeon & Park, Yongtae & Choi, Hoogon, 2009. "Comparative evaluation of performance of national R&D programs with heterogeneous objectives: A DEA approach," European Journal of Operational Research, Elsevier, vol. 196(3), pages 847-855, August.
    32. Hassanzadeh, Farhad & Nemati, Hamid & Sun, Minghe, 2014. "Robust optimization for interactive multiobjective programming with imprecise information applied to R&D project portfolio selection," European Journal of Operational Research, Elsevier, vol. 238(1), pages 41-53.
    33. Li, Xingmei & Huang, Yao-Huei & Fang, Shu-Cherng & Zhang, Youzhong, 2020. "An alternative efficient representation for the project portfolio selection problem," European Journal of Operational Research, Elsevier, vol. 281(1), pages 100-113.
    34. Fliedner, Thomas & Liesiö, Juuso, 2016. "Adjustable robustness for multi-attribute project portfolio selection," European Journal of Operational Research, Elsevier, vol. 252(3), pages 931-946.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gasparini, Gaia & Brunelli, Matteo & Chiriac, Marius Dan, 2022. "Multi-period portfolio decision analysis: A case study in the infrastructure management sector," Operations Research Perspectives, Elsevier, vol. 9(C).
    2. Hongbo Li & Rui Chen & Xianchao Zhang, 2022. "Uncertain Public R&D Project Portfolio Selection Considering Sectoral Balancing and Project Failure," Sustainability, MDPI, vol. 14(23), pages 1-13, November.
    3. Selin Özpeynirci & Özgür Özpeynirci & Vincent Mousseau, 2021. "An interactive algorithm for resource allocation with balance concerns," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 43(4), pages 983-1005, December.
    4. George Mavrotas & Evangelos Makryvelios, 2023. "R&D project portfolio selection using the Iterative Trichotomic Approach in order to study how subjectivity of the weights is reflected in the selected projects of the final portfolio," Operational Research, Springer, vol. 23(3), pages 1-18, September.
    5. Jones, Dylan & Labib, Ashraf & Willis, Kevin & Costello, Joseph T & Ouelhadj, Djamila & Ikonen, Emmi Susanna & Dominguez Cainzos, Mikel, 2023. "Multi-criteria mapping and prioritization of Arctic and North Atlantic maritime safety and security needs," European Journal of Operational Research, Elsevier, vol. 307(2), pages 827-841.
    6. Fernández, Eduardo & Figueira, José Rui & Navarro, Jorge & Solares, Efrain, 2022. "Handling imperfect information in multiple criteria decision-making through a comprehensive interval outranking approach," Socio-Economic Planning Sciences, Elsevier, vol. 82(PB).
    7. Wenfeng Zhu & Hengjie Zhang & Jing Xiao, 2023. "Coming to Consensus on Classification in Flexible Linguistic Preference Relations: The Role of Personalized Individual Semantics," Group Decision and Negotiation, Springer, vol. 32(5), pages 1237-1271, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Liesiö, Juuso & Salo, Ahti & Keisler, Jeffrey M. & Morton, Alec, 2021. "Portfolio decision analysis: Recent developments and future prospects," European Journal of Operational Research, Elsevier, vol. 293(3), pages 811-825.
    2. Panos Xidonas & Haris Doukas & George Mavrotas & Olena Pechak, 2016. "Environmental corporate responsibility for investments evaluation: an alternative multi-objective programming model," Annals of Operations Research, Springer, vol. 247(2), pages 395-413, December.
    3. Marttunen, Mika & Haara, Arto & Hjerppe, Turo & Kurttila, Mikko & Liesiö, Juuso & Mustajoki, Jyri & Saarikoski, Heli & Tolvanen, Anne, 2023. "Parallel and comparative use of three multicriteria decision support methods in an environmental portfolio problem," European Journal of Operational Research, Elsevier, vol. 307(2), pages 842-859.
    4. George Mavrotas & Evangelos Makryvelios, 2023. "R&D project portfolio selection using the Iterative Trichotomic Approach in order to study how subjectivity of the weights is reflected in the selected projects of the final portfolio," Operational Research, Springer, vol. 23(3), pages 1-18, September.
    5. Barbati, M. & Figueira, J.R. & Greco, S. & Ishizaka, A. & Panaro, S., 2023. "A multiple criteria methodology for priority based portfolio selection," Socio-Economic Planning Sciences, Elsevier, vol. 88(C).
    6. Mavrotas, George & Diakoulaki, Danae & Kourentzis, Athanasios, 2008. "Selection among ranked projects under segmentation, policy and logical constraints," European Journal of Operational Research, Elsevier, vol. 187(1), pages 177-192, May.
    7. Milford, James & Henrion, Max & Hunter, Chad & Newes, Emily & Hughes, Caroline & Baldwin, Samuel F., 2022. "Energy sector portfolio analysis with uncertainty," Applied Energy, Elsevier, vol. 306(PA).
    8. Vilkkumaa, Eeva & Liesiö, Juuso & Salo, Ahti & Ilmola-Sheppard, Leena, 2018. "Scenario-based portfolio model for building robust and proactive strategies," European Journal of Operational Research, Elsevier, vol. 266(1), pages 205-220.
    9. Barbati, Maria & Greco, Salvatore & Kadziński, Miłosz & Słowiński, Roman, 2018. "Optimization of multiple satisfaction levels in portfolio decision analysis," Omega, Elsevier, vol. 78(C), pages 192-204.
    10. Marques, Adriana Cavalcante & Frej, Eduarda Asfora & de Almeida, Adiel Teixeira, 2022. "Multicriteria decision support for project portfolio selection with the FITradeoff method," Omega, Elsevier, vol. 111(C).
    11. Baker, Erin & Bosetti, Valentina & Salo, Ahti, 2020. "Robust portfolio decision analysis: An application to the energy research and development portfolio problem," European Journal of Operational Research, Elsevier, vol. 284(3), pages 1107-1120.
    12. Selin Özpeynirci & Özgür Özpeynirci & Vincent Mousseau, 2021. "An interactive algorithm for resource allocation with balance concerns," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 43(4), pages 983-1005, December.
    13. Fliedner, Thomas & Liesiö, Juuso, 2016. "Adjustable robustness for multi-attribute project portfolio selection," European Journal of Operational Research, Elsevier, vol. 252(3), pages 931-946.
    14. Vilkkumaa, Eeva & Liesiö, Juuso & Salo, Ahti, 2014. "Optimal strategies for selecting project portfolios using uncertain value estimates," European Journal of Operational Research, Elsevier, vol. 233(3), pages 772-783.
    15. Toppila, Antti & Salo, Ahti, 2017. "Binary decision diagrams for generating and storing non-dominated project portfolios with interval-valued project scores," European Journal of Operational Research, Elsevier, vol. 260(1), pages 244-254.
    16. Pérez, Fátima & Gómez, Trinidad & Caballero, Rafael & Liern, Vicente, 2018. "Project portfolio selection and planning with fuzzy constraints," Technological Forecasting and Social Change, Elsevier, vol. 131(C), pages 117-129.
    17. Rudolf vetschera & Jonatas Araùjo de Almeida, 2021. "Bounds in Tree-Based Approaches to Generate Project Portfolios in the Presence of Interactions," International Journal of Decision Support System Technology (IJDSST), IGI Global, vol. 13(4), pages 1-21, October.
    18. Çağlar, Musa & Gürel, Sinan, 2019. "Impact assessment based sectoral balancing in public R&D project portfolio selection," Socio-Economic Planning Sciences, Elsevier, vol. 66(C), pages 68-81.
    19. Antti Punkka & Ahti Salo, 2014. "Scale Dependence and Ranking Intervals in Additive Value Models Under Incomplete Preference Information," Decision Analysis, INFORMS, vol. 11(2), pages 83-104, June.
    20. Schäfer, Luca E. & Dietz, Tobias & Barbati, Maria & Figueira, José Rui & Greco, Salvatore & Ruzika, Stefan, 2021. "The binary knapsack problem with qualitative levels," European Journal of Operational Research, Elsevier, vol. 289(2), pages 508-514.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:291:y:2021:i:2:p:794-806. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.