IDEAS home Printed from https://ideas.repec.org/a/bpj/jqsprt/v12y2016i4p151-165n3.html
   My bibliography  Save this article

Meta-analytics: tools for understanding the statistical properties of sports metrics

Author

Listed:
  • Franks Alexander M.

    (University of Washington - Department of Statistics, Seattle, WA, USA)

  • D’Amour Alexander

    (University of Berkeley - Department of Statistics, Berkeley, CA, USA)

  • Cervone Daniel

    (New York University, New York, NY, USA)

  • Bornn Luke

    (Simon Fraser University - Department of Statistics, Burnaby, British Columbia, Canada)

Abstract

In sports, there is a constant effort to improve metrics that assess player ability, but there has been almost no effort to quantify and compare existing metrics. Any individual making a management, coaching, or gambling decision is quickly overwhelmed with hundreds of statistics. We address this problem by proposing a set of “meta-metrics”, which can be used to identify the metrics that provide the most unique and reliable information for decision-makers. Specifically, we develop methods to evaluate metrics based on three criteria: (1) stability: does the metric measure the same thing over time (2) discrimination: does the metric differentiate between players and (3) independence: does the metric provide new information? Our methods are easy to implement and widely applicable so they should be of interest to the broader sports community. We demonstrate our methods in analyses of both NBA and NHL metrics. Our results indicate the most reliable metrics and highlight how they should be used by sports analysts. The meta-metrics also provide useful insights about how to best construct new metrics that provide independent and reliable information about athletes.

Suggested Citation

  • Franks Alexander M. & D’Amour Alexander & Cervone Daniel & Bornn Luke, 2016. "Meta-analytics: tools for understanding the statistical properties of sports metrics," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 12(4), pages 151-165, December.
    • Handle: RePEc:bpj:jqsprt:v:12:y:2016:i:4:p:151-165:n:3
    DOI: 10.1515/jqas-2016-0098
    as

    Download full text from publisher

    File URL: https://doi.org/10.1515/jqas-2016-0098
    Download Restriction: For access to full text, subscription to the journal or payment for the individual article is required.
    File URL: https://libkey.io/10.1515/jqas-2016-0098?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bradley Efron, 2015. "Frequentist accuracy of Bayesian estimates," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 77(3), pages 617-646, June.
    2. Baumer Benjamin S. & Jensen Shane T. & Matthews Gregory J., 2015. "openWAR: An open source system for evaluating overall player performance in major league baseball," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 11(2), pages 69-84, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mallepalle Sarah & Yurko Ronald & Ventura Samuel L. & Pelechrinis Konstantinos, 2020. "Extracting NFL tracking data from images to evaluate quarterbacks and pass defenses," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 16(2), pages 95-120, June.
    2. Marco Sandri & Paola Zuccolotto & Marica Manisera, 2020. "Markov switching modelling of shooting performance variability and teammate interactions in basketball," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 69(5), pages 1337-1356, November.
    3. Yurko Ronald & Ventura Samuel & Horowitz Maksim, 2019. "nflWAR: a reproducible method for offensive player evaluation in football," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 15(3), pages 163-183, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Matthew Reimherr & Xiao‐Li Meng & Dan L. Nicolae, 2021. "Prior sample size extensions for assessing prior impact and prior‐likelihood discordance," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(3), pages 413-437, July.
    2. Chu Dani & Reyers Matthew & Thomson James & Wu Lucas Yifan, 2020. "Route identification in the National Football League: An application of model-based curve clustering using the EM algorithm," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 16(2), pages 121-132, June.
    3. Mallepalle Sarah & Yurko Ronald & Ventura Samuel L. & Pelechrinis Konstantinos, 2020. "Extracting NFL tracking data from images to evaluate quarterbacks and pass defenses," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 16(2), pages 95-120, June.
    4. Kleijnen, Jack & van Nieuwenhuyse, I. & van Beers, W.C.M., 2022. "Constrained Optimization in Simulation : Efficient Global Optimization and Karush-Kuhn-Tucker Conditions (revision of 2021-031)," Other publications TiSEM 31a06a3b-dfc4-4431-a141-5, Tilburg University, School of Economics and Management.
    5. Timothy B. Armstrong & Michal Kolesár & Mikkel Plagborg‐Møller, 2022. "Robust Empirical Bayes Confidence Intervals," Econometrica, Econometric Society, vol. 90(6), pages 2567-2602, November.
    6. De Luca, Giuseppe & Magnus, Jan R. & Peracchi, Franco, 2022. "Sampling properties of the Bayesian posterior mean with an application to WALS estimation," Journal of Econometrics, Elsevier, vol. 230(2), pages 299-317.
    7. Kleijnen, Jack P.C., 2017. "Regression and Kriging metamodels with their experimental designs in simulation: A review," European Journal of Operational Research, Elsevier, vol. 256(1), pages 1-16.
    8. Ferreira, Marco A.R. & Porter, Erica M. & Franck, Christopher T., 2021. "Fast and scalable computations for Gaussian hierarchical models with intrinsic conditional autoregressive spatial random effects," Computational Statistics & Data Analysis, Elsevier, vol. 162(C).
    9. Yurko Ronald & Ventura Samuel & Horowitz Maksim, 2019. "nflWAR: a reproducible method for offensive player evaluation in football," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 15(3), pages 163-183, September.
    10. Jyh-How Huang & Yu-Chia Hsu, 2021. "A Multidisciplinary Perspective on Publicly Available Sports Data in the Era of Big Data: A Scoping Review of the Literature on Major League Baseball," SAGE Open, , vol. 11(4), pages 21582440211, November.
    11. Yurko Ronald & Ventura Samuel & Horowitz Maksim, 2019. "nflWAR: a reproducible method for offensive player evaluation in football," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 15(3), pages 163-183, September.
    12. Kleijnen, Jack & van Beers, W.C.M., 2019. "Statistical Tests for Cross-Validation of Kriging Models," Other publications TiSEM 35fba511-2931-47d5-a9ba-3, Tilburg University, School of Economics and Management.
    13. Jack P. C. Kleijnen & Wim C. M. van Beers, 2022. "Statistical Tests for Cross-Validation of Kriging Models," INFORMS Journal on Computing, INFORMS, vol. 34(1), pages 607-621, January.
    14. Shane Sanders & Joel Potter & Justin Ehrlich & Justin Perline & Christopher Boudreaux, 2021. "Informed voters and electoral outcomes: a natural experiment stemming from a fundamental information-technological shift," Public Choice, Springer, vol. 189(1), pages 257-277, October.
    15. Timothy B. Armstrong & Michal Koles'ar & Mikkel Plagborg-M{o}ller, 2020. "Robust Empirical Bayes Confidence Intervals," Papers 2004.03448, arXiv.org, revised May 2022.
    16. DongHyuk Lee & Raymond J. Carroll & Samiran Sinha, 2017. "Frequentist standard errors of Bayes estimators," Computational Statistics, Springer, vol. 32(3), pages 867-888, September.
    17. Vock David Michael & Vock Laura Frances Boehm, 2018. "Estimating the effect of plate discipline using a causal inference framework: an application of the G-computation algorithm," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 14(2), pages 37-56, June.
    18. Elsa Vazquez & Jeffrey R. Wilson, 2021. "Partitioned method of valid moment marginal model with Bayes interval estimates for correlated binary data with time-dependent covariates," Computational Statistics, Springer, vol. 36(4), pages 2701-2718, December.
    19. Andres Ramirez-Hassan & Manuel Correa-Giraldo, 2018. "Focused econometric estimation for noisy and small datasets: A Bayesian Minimum Expected Loss estimator approach," Papers 1809.06996, arXiv.org.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:jqsprt:v:12:y:2016:i:4:p:151-165:n:3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.degruyter.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.