IDEAS home Printed from https://ideas.repec.org/a/spr/jcomop/v31y2016i2d10.1007_s10878-014-9784-3.html

Team selection for prediction tasks

Author

Listed:
  • MohammadAmin Fazli

    (Sharif University of Technology)

  • Azin Ghazimatin

    (Sharif University of Technology)

  • Jafar Habibi

    (Sharif University of Technology)

  • Hamid Haghshenas

    (Sharif University of Technology)

Abstract

Given a random variable $$O \in \mathbb {R}$$ O ∈ R and a set of experts $$E$$ E , we describe a method for finding a subset of experts $$S \subseteq E$$ S ⊆ E whose aggregated opinion best predicts the outcome of $$O$$ O . Therefore, the problem can be regarded as a team formation for performing a prediction task. We show that in case of aggregating experts’ opinions by simple averaging, finding the best team (the team with the lowest total error during past $$k$$ k rounds) can be modeled with an integer quadratic programming and we prove its NP-hardness whereas its relaxation is solvable in polynomial time. At the end, we do an experimental comparison between different rounding and greedy heuristics on artificial datasets which are generated based on calibration and informativeness of exprets’ information and show that our suggested tabu search works effectively.

Suggested Citation

  • MohammadAmin Fazli & Azin Ghazimatin & Jafar Habibi & Hamid Haghshenas, 2016. "Team selection for prediction tasks," Journal of Combinatorial Optimization, Springer, vol. 31(2), pages 743-757, February.
    • Handle: RePEc:spr:jcomop:v:31:y:2016:i:2:d:10.1007_s10878-014-9784-3
    DOI: 10.1007/s10878-014-9784-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10878-014-9784-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10878-014-9784-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Yiling Chen & Chao-Hsien Chu & Tracy Mullen, 2006. "Predicting Uncertain Outcomes Using Information Markets: Trader Behavior And Information Aggregation," New Mathematics and Natural Computation (NMNC), World Scientific Publishing Co. Pte. Ltd., vol. 2(03), pages 281-297.
    2. Peter A. Morris, 1974. "Decision Analysis Expert Use," Management Science, INFORMS, vol. 20(9), pages 1233-1241, May.
    3. James K. Hammitt & Yifan Zhang, 2013. "Combining Experts’ Judgments: Comparison of Algorithmic Methods Using Synthetic Data," Risk Analysis, John Wiley & Sons, vol. 33(1), pages 109-120, January.
    4. Qinghua Wu & Jin-Kao Hao, 2013. "An adaptive multistart tabu search approach to solve the maximum clique problem," Journal of Combinatorial Optimization, Springer, vol. 26(1), pages 86-108, July.
    5. Robert T. Clemen & Robert L. Winkler, 1999. "Combining Probability Distributions From Experts in Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 19(2), pages 187-203, April.
    6. Mostaghimi, Mehdi, 1996. "Combining ranked mean value forecasts," European Journal of Operational Research, Elsevier, vol. 94(3), pages 505-516, November.
    7. Graefe, Andreas & Armstrong, J. Scott & Jones, Randall J. & Cuzán, Alfred G., 2014. "Combining forecasts: An application to elections," International Journal of Forecasting, Elsevier, vol. 30(1), pages 43-54.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jason R. W. Merrick, 2008. "Getting the Right Mix of Experts," Decision Analysis, INFORMS, vol. 5(1), pages 43-52, March.
    2. Wang, Xiaoqian & Hyndman, Rob J. & Li, Feng & Kang, Yanfei, 2023. "Forecast combinations: An over 50-year review," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1518-1547.
    3. Neil A. Stiber & Mitchell J. Small & Marina Pantazidou, 2004. "Site‐Specific Updating and Aggregation of Bayesian Belief Network Models for Multiple Experts," Risk Analysis, John Wiley & Sons, vol. 24(6), pages 1529-1538, December.
    4. Robert L. Winkler & Yael Grushka-Cockayne & Kenneth C. Lichtendahl Jr. & Victor Richmond R. Jose, 2019. "Probability Forecasts and Their Combination: A Research Perspective," Decision Analysis, INFORMS, vol. 16(4), pages 239-260, December.
    5. Jason R. W. Merrick & J. Rene van Dorp & Amita Singh, 2005. "Analysis of Correlated Expert Judgments from Extended Pairwise Comparisons," Decision Analysis, INFORMS, vol. 2(1), pages 17-29, March.
    6. Patrick Afflerbach & Christopher Dun & Henner Gimpel & Dominik Parak & Johannes Seyfried, 2021. "A Simulation-Based Approach to Understanding the Wisdom of Crowds Phenomenon in Aggregating Expert Judgment," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 63(4), pages 329-348, August.
    7. repec:ces:ceswps:_11668 is not listed on IDEAS
    8. Ali E. Abbas, 2009. "A Kullback-Leibler View of Linear and Log-Linear Pools," Decision Analysis, INFORMS, vol. 6(1), pages 25-37, March.
    9. Julia R. Falconer & Eibe Frank & Devon L. L. Polaschek & Chaitanya Joshi, 2022. "Methods for Eliciting Informative Prior Distributions: A Critical Review," Decision Analysis, INFORMS, vol. 19(3), pages 189-204, September.
    10. Christopher W. Karvetski & David R. Mandel & Daniel Irwin, 2020. "Improving Probability Judgment in Intelligence Analysis: From Structured Analysis to Statistical Aggregation," Risk Analysis, John Wiley & Sons, vol. 40(5), pages 1040-1057, May.
    11. Knut Are Aastveit & James Mitchell & Francesco Ravazzolo & Herman van Dijk, 2018. "The Evolution of Forecast Density Combinations in Economics," Tinbergen Institute Discussion Papers 18-069/III, Tinbergen Institute.
    12. Armstrong, J. Scott & Green, Kesten C. & Graefe, Andreas, 2015. "Golden rule of forecasting: Be conservative," Journal of Business Research, Elsevier, vol. 68(8), pages 1717-1731.
    13. Avner Engel & Shalom Shachar, 2006. "Measuring and optimizing systems' quality costs and project duration," Systems Engineering, John Wiley & Sons, vol. 9(3), pages 259-280, September.
    14. Hall, Stephen G. & Mitchell, James, 2007. "Combining density forecasts," International Journal of Forecasting, Elsevier, vol. 23(1), pages 1-13.
    15. Ali Mosleh & George Apostolakis, 1986. "The Assessment of Probability Distributions from Expert Opinions with an Application to Seismic Fragility Curves," Risk Analysis, John Wiley & Sons, vol. 6(4), pages 447-461, December.
    16. Thibault Gajdos & Jean-Christophe Vergnaud, 2013. "Decisions with conflicting and imprecise information," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 41(2), pages 427-452, July.
    17. Elkin Casta�o V. & Luis Fernando Melo Velandia, 1998. "M�Todos De Combinaci�N De Pron�Sticos:Una Aplicaci�N A La Inflaci�N Colombiana," Borradores de Economia 3212, Banco de la Republica.
    18. Bohdan M. Pavlyshenko, 2019. "Machine-Learning Models for Sales Time Series Forecasting," Data, MDPI, vol. 4(1), pages 1-11, January.
    19. Khan, Urmee & Lieli, Robert P., 2018. "Information flow between prediction markets, polls and media: Evidence from the 2008 presidential primaries," International Journal of Forecasting, Elsevier, vol. 34(4), pages 696-710.
    20. Cui, Mengying & Yu, Lijie & Nie, Shaoyu & Dai, Zhe & Ge, Ying-en & Levinson, David, 2025. "How do access and spatial dependency shape metro passenger flows," Journal of Transport Geography, Elsevier, vol. 123(C).
    21. Robert L. Winkler & Robert T. Clemen, 2004. "Multiple Experts vs. Multiple Methods: Combining Correlation Assessments," Decision Analysis, INFORMS, vol. 1(3), pages 167-176, September.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:jcomop:v:31:y:2016:i:2:d:10.1007_s10878-014-9784-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.