IDEAS home Printed from https://ideas.repec.org/a/gam/jforec/v7y2025i1p4-d1566045.html
   My bibliography  Save this article

Testing for Bias in Forecasts for Independent Multinomial Outcomes

Author

Listed:
  • Philip Hans Franses

    (Econometric Institute, Erasmus School of Economics, Burgemeester Oudlaan 50, 3062PA Rotterdam, The Netherlands)

  • Richard Paap

    (Econometric Institute, Erasmus School of Economics, Burgemeester Oudlaan 50, 3062PA Rotterdam, The Netherlands)

Abstract

This paper deals with a test on forecast bias in predicting independent multinomial outcomes where the predictions are probabilities. The new Likelihood Ratio (and Wald) test extends the familiar Mincer Zarnowitz regression to a multinomial logit model instead of a linear regression. The test is evaluated using various simulation experiments, which indicate that the size and power properties are good, even for small sample sizes, in the sense that the size is close to the used 5% level, and the power quickly reaches 1. We implement the test in an empirical setting on brand choice by individual households.

Suggested Citation

  • Philip Hans Franses & Richard Paap, 2025. "Testing for Bias in Forecasts for Independent Multinomial Outcomes," Forecasting, MDPI, vol. 7(1), pages 1-8, January.
    • Handle: RePEc:gam:jforec:v:7:y:2025:i:1:p:4-:d:1566045
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2571-9394/7/1/4/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2571-9394/7/1/4/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Franses,Philip Hans & Paap,Richard, 2010. "Quantitative Models in Marketing Research," Cambridge Books, Cambridge University Press, number 9780521143653, Enero-Abr.
    2. Jacob A. Mincer & Victor Zarnowitz, 1969. "The Evaluation of Economic Forecasts," NBER Chapters, in: Economic Forecasts and Expectations: Analysis of Forecasting Behavior and Performance, pages 3-46, National Bureau of Economic Research, Inc.
    3. Anonymous, 1969. "Chapter II. Forecasts for the Economy," National Institute Economic Review, National Institute of Economic and Social Research, vol. 47, pages 32-46, February.
    4. Philip Hans Franses, 2021. "Testing for bias in forecasts for independent binary outcomes," Applied Economics Letters, Taylor & Francis Journals, vol. 28(15), pages 1336-1338, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Risselada, Hans & Verhoef, Peter C. & Bijmolt, Tammo H.A., 2010. "Staying Power of Churn Prediction Models," Journal of Interactive Marketing, Elsevier, vol. 24(3), pages 198-208.
    2. Hendry, David F. & Clements, Michael P., 2003. "Economic forecasting: some lessons from recent research," Economic Modelling, Elsevier, vol. 20(2), pages 301-329, March.
    3. Graham Elliott & Ivana Komunjer & Allan Timmermann, 2008. "Biases in Macroeconomic Forecasts: Irrationality or Asymmetric Loss?," Journal of the European Economic Association, MIT Press, vol. 6(1), pages 122-157, March.
    4. Polo, Yolanda & Sese, F. Javier & Verhoef, Peter C., 2011. "The Effect of Pricing and Advertising on Customer Retention in a Liberalizing Market," Journal of Interactive Marketing, Elsevier, vol. 25(4), pages 201-214.
    5. Jan Jacobs & Jan-Egbert Sturm, 2009. "The information content of KOF indicators on Swiss current account data revisions," OECD Journal: Journal of Business Cycle Measurement and Analysis, OECD Publishing, Centre for International Research on Economic Tendency Surveys, vol. 2008(2), pages 161-181.
    6. repec:kap:iaecre:v:14:y:2008:i:1:p:112-124 is not listed on IDEAS
    7. Barbara Rossi, 2013. "Exchange Rate Predictability," Journal of Economic Literature, American Economic Association, vol. 51(4), pages 1063-1119, December.
    8. Pericoli, Marcello & Taboga, Marco, 2012. "Bond risk premia, macroeconomic fundamentals and the exchange rate," International Review of Economics & Finance, Elsevier, vol. 22(1), pages 42-65.
    9. Chang, Andrew C. & Hanson, Tyler J., 2016. "The accuracy of forecasts prepared for the Federal Open Market Committee," Journal of Economics and Business, Elsevier, vol. 83(C), pages 23-43.
    10. Sucarrat, Genaro, 2009. "Forecast Evaluation of Explanatory Models of Financial Variability," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 3, pages 1-33.
    11. Michael P. Clements, 2014. "US Inflation Expectations and Heterogeneous Loss Functions, 1968–2010," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 33(1), pages 1-14, January.
    12. Bespalova, Olga, 2018. "Forecast Evaluation in Macroeconomics and International Finance. Ph.D. thesis, George Washington University, Washington, DC, USA," MPRA Paper 117706, University Library of Munich, Germany.
    13. Edwin Van Gameren & Michiel Ras & Evelien Eggink & Ingrid Ooms, 2005. "The demand for housing services in the Netherlands," ERSA conference papers ersa05p327, European Regional Science Association.
    14. Rituparna Sen & Pulkit Mehrotra, 2016. "Modeling Jumps and Volatility of the Indian Stock Market Using High-Frequency Data," Journal of Quantitative Economics, Springer;The Indian Econometric Society (TIES), vol. 14(1), pages 137-150, June.
    15. Franses, Philip Hans, 2006. "Forecasting in Marketing," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 18, pages 983-1012, Elsevier.
    16. Döpke, Jörg & Fritsche, Ulrich & Müller, Karsten, 2019. "Has macroeconomic forecasting changed after the Great Recession? Panel-based evidence on forecast accuracy and forecaster behavior from Germany," Journal of Macroeconomics, Elsevier, vol. 62(C).
    17. Philip Hans Franses & Max Welz, 2022. "Evaluating heterogeneous forecasts for vintages of macroeconomic variables," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(4), pages 829-839, July.
    18. Patrick Rizzetto, 2018. "GDP by Industry in Real Time: Are Revisions Well Behaved?," Staff Analytical Notes 2018-40, Bank of Canada.
    19. Rodríguez-Vargas, Adolfo, 2020. "Forecasting Costa Rican inflation with machine learning methods," Latin American Journal of Central Banking (previously Monetaria), Elsevier, vol. 1(1).
    20. Ceci, Vladimiro & Manganelli, Simone & Vecchiato, Walter, 2002. "Sensitivity analysis of volatility: a new tool for risk management," Working Paper Series 194, European Central Bank.
    21. Benchimol, Jonathan & El-Shagi, Makram & Saadon, Yossi, 2022. "Do expert experience and characteristics affect inflation forecasts?," Journal of Economic Behavior & Organization, Elsevier, vol. 201(C), pages 205-226.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jforec:v:7:y:2025:i:1:p:4-:d:1566045. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.