IDEAS home Printed from https://ideas.repec.org/a/sae/jospec/v26y2025i1p3-34.html

Ranking in Incomplete Tournaments: The Generalized Win Percentage Method, Efficiency, and NCAA Football

Author

Listed:
  • Fernando Leiva Bertrán

Abstract

I present a parsimonious family of scoring methods for ranking teams in incomplete tournaments. It is especially useful if teams play different number of games. I show that this family of scoring methods satisfies common sense theoretical properties studied by the literature on paired comparisons. I analyze it in terms of efficiency, defined as how close a scoring method comes to capturing what the teams’ win percentages would have been, in a complete tournament. Finally, using data on betting odds, I calibrate the family of scoring methods to match, as closely as possible, the actual rankings that were used to determine the teams that would go on to compete for the championship of the NCAA division 1 football tournament between 2011 and 2023. I find that the rankings used by the NCAA were generally efficient and I quantify the biases present in each year’s ranking.

Suggested Citation

  • Fernando Leiva Bertrán, 2025. "Ranking in Incomplete Tournaments: The Generalized Win Percentage Method, Efficiency, and NCAA Football," Journal of Sports Economics, , vol. 26(1), pages 3-34, January.
    • Handle: RePEc:sae:jospec:v:26:y:2025:i:1:p:3-34
    DOI: 10.1177/15270025241268738
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/15270025241268738
    Download Restriction: no
    File URL: https://libkey.io/10.1177/15270025241268738?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Scarf, Philip & Yusof, Muhammad Mat & Bilbao, Mark, 2009. "A numerical study of designs for sporting contests," European Journal of Operational Research, Elsevier, vol. 198(1), pages 190-198, October.
    2. Can, Burak, 2014. "Weighted distances between preferences," Journal of Mathematical Economics, Elsevier, vol. 51(C), pages 109-115.
    3. Devlin Stephen & Treloar Thomas, 2018. "A network diffusion ranking family that includes the methods of Markov, Massey, and Colley," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 14(3), pages 91-101, September.
    4. Čaklović, Lavoslav & Kurdija, Adrian Satja, 2017. "A universal voting system based on the Potential Method," European Journal of Operational Research, Elsevier, vol. 259(2), pages 677-688.
    5. Baback Vaziri & Shaunak Dabadghao & Yuehwern Yih & Thomas L. Morin, 2018. "Properties of sports ranking methods," Journal of the Operational Research Society, Taylor & Francis Journals, vol. 69(5), pages 776-787, May.
    6. Csató, László, 2023. "How to avoid uncompetitive games? The importance of tie-breaking rules," European Journal of Operational Research, Elsevier, vol. 307(3), pages 1260-1269.
    7. Giora Slutzki & Oscar Volij, 2006. "Scoring of web pages and tournaments—axiomatizations," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 26(1), pages 75-92, January.
    8. Sauer Pascal & Cseh Ágnes & Lenzner Pascal, 2024. "Improving ranking quality and fairness in Swiss-system chess tournaments," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 20(2), pages 127-146, June.
    9. Ignacio Palacios-Huerta & Oscar Volij, 2004. "The Measurement of Intellectual Influence," Econometrica, Econometric Society, vol. 72(3), pages 963-977, May.
    10. László Csató, 2013. "Ranking by pairwise comparisons for Swiss-system tournaments," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 21(4), pages 783-803, December.
    11. Pavel Yu. Chebotarev & Elena Shamis, 1998. "Characterizations of scoring methodsfor preference aggregation," Annals of Operations Research, Springer, vol. 80(0), pages 299-332, January.
    12. András Gyimesi, 2024. "Competitive Balance in the Post-2024 Champions League and the European Super League: A Simulation Study," Journal of Sports Economics, , vol. 25(6), pages 707-734, August.
    13. Scarf, Phil & Parma, Rishikesh & McHale, Ian, 2019. "On outcome uncertainty and scoring rates in sport: The case of international rugby union," European Journal of Operational Research, Elsevier, vol. 273(2), pages 721-730.
    14. László Csató, 2019. "An impossibility theorem for paired comparisons," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 27(2), pages 497-514, June.
    15. Bozóki, Sándor & Csató, László & Temesi, József, 2016. "An application of incomplete pairwise comparison matrices for ranking top tennis players," European Journal of Operational Research, Elsevier, vol. 248(1), pages 211-218.
    16. Csató, László & Tóth, Csaba, 2020. "University rankings from the revealed preferences of the applicants," European Journal of Operational Research, Elsevier, vol. 286(1), pages 309-320.
    17. Hans Eetvelde & Lars Magnus Hvattum & Christophe Ley, 2023. "The Probabilistic Final Standing Calculator: a fair stochastic tool to handle abruptly stopped football seasons," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 107(1), pages 251-269, March.
    18. S. S. Dabadghao & B. Vaziri, 2022. "The predictive power of popular sports ranking methods in the NFL, NBA, and NHL," Operational Research, Springer, vol. 22(3), pages 2767-2783, July.
    19. László Csató, 2022. "UEFA Against the Champions? An Evaluation of the Recent Reform of the Champions League Qualification," Journal of Sports Economics, , vol. 23(8), pages 991-1016, December.
    20. Julio González-Díaz & Ruud Hendrickx & Edwin Lohmann, 2014. "Paired comparisons analysis: an axiomatic approach to ranking methods," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 42(1), pages 139-169, January.
    21. Ausloos, Marcel, 2024. "Hierarchy selection: New team ranking indicators for cyclist multi-stage races," European Journal of Operational Research, Elsevier, vol. 314(2), pages 807-816.
    22. László Csató, 2017. "On the ranking of a Swiss system chess team tournament," Annals of Operations Research, Springer, vol. 254(1), pages 17-36, July.
    23. P. Gorgi & S. J. Koopman & R. Lit, 2023. "Estimation of final standings in football competitions with a premature ending: the case of COVID-19," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 107(1), pages 233-250, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Devriesere, Karel & Csató, László & Goossens, Dries, 2025. "Tournament design: A review from an operational research perspective," European Journal of Operational Research, Elsevier, vol. 324(1), pages 1-21.
    2. Csató, László & Tóth, Csaba, 2020. "University rankings from the revealed preferences of the applicants," European Journal of Operational Research, Elsevier, vol. 286(1), pages 309-320.
    3. L'aszl'o Csat'o & Csaba T'oth, 2018. "University rankings from the revealed preferences of the applicants," Papers 1810.04087, arXiv.org, revised Feb 2020.
    4. Han, Weibin & Deemen, Adrian Van, 2025. "The Copeland ratio ranking method for abstract decision problems," European Journal of Operational Research, Elsevier, vol. 323(3), pages 966-974.
    5. Petróczy, Dóra Gréta, 2021. "An alternative quality of life ranking on the basis of remittances," Socio-Economic Planning Sciences, Elsevier, vol. 78(C).
    6. László Csató, 2019. "An impossibility theorem for paired comparisons," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 27(2), pages 497-514, June.
    7. Éva Orbán-Mihálykó & Csaba Mihálykó & László Gyarmati, 2024. "Evaluating the capacity of paired comparison methods to aggregate rankings of separate groups," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 32(1), pages 109-129, March.
    8. Szádoczki, Zsombor & Bozóki, Sándor & Tekile, Hailemariam Abebe, 2022. "Filling in pattern designs for incomplete pairwise comparison matrices: (Quasi-)regular graphs with minimal diameter," Omega, Elsevier, vol. 107(C).
    9. Ágoston, Kolos Csaba & Csató, László, 2022. "Inconsistency thresholds for incomplete pairwise comparison matrices," Omega, Elsevier, vol. 108(C).
    10. Ausloos, Marcel, 2024. "Hierarchy selection: New team ranking indicators for cyclist multi-stage races," European Journal of Operational Research, Elsevier, vol. 314(2), pages 807-816.
    11. László Csató, 2017. "On the ranking of a Swiss system chess team tournament," Annals of Operations Research, Springer, vol. 254(1), pages 17-36, July.
    12. L'aszl'o Csat'o, 2025. "A hidden benefit of incomplete round-robin tournaments: Encouraging offensive play," Papers 2509.13141, arXiv.org.
    13. Fernandes, Rosário & Furtado, Susana, 2022. "Efficiency of the principal eigenvector of some triple perturbed consistent matrices," European Journal of Operational Research, Elsevier, vol. 298(3), pages 1007-1015.
    14. Csató, László & Petróczy, Dóra Gréta, 2021. "On the monotonicity of the eigenvector method," European Journal of Operational Research, Elsevier, vol. 292(1), pages 230-237.
    15. Raphael Flepp & Tim Pawlowski & Travis Richardson, 2025. "Suspense and Surprise in European Football," Papers 2506.21253, arXiv.org.
    16. Albarrán, Pedro & Herrero, Carmen & Ruiz-Castillo, Javier & Villar, Antonio, 2017. "The Herrero-Villar approach to citation impact," Journal of Informetrics, Elsevier, vol. 11(2), pages 625-640.
    17. Csató, László, 2019. "Journal ranking should depend on the level of aggregation," Journal of Informetrics, Elsevier, vol. 13(4).
    18. L'aszl'o Csat'o & Andr'as Gyimesi, 2026. "A probabilistic match classification model for sports tournaments," Papers 2601.09673, arXiv.org, revised Jan 2026.
    19. Ritxar Arlegi & Dinko Dimitrov, 2023. "League competitions and fairness," Journal of Combinatorial Optimization, Springer, vol. 45(4), pages 1-18, May.
    20. Carmen Herrero & Antonio Villar, 2013. "On the Comparison of Group Performance with Categorical Data," PLOS ONE, Public Library of Science, vol. 8(12), pages 1-7, December.

    More about this item

    Keywords

    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jospec:v:26:y:2025:i:1:p:3-34. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.