IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2608.23808.html

Equity Strategy Backtesting: Luck or Edge? The MinervaScore as a Statistical Robustness Grade

Author

Listed:
  • Maria Laura Santoni
  • Vincent Jouanne
  • Matthew L. Scullin

Abstract

Backtests of trading strategies are often selected after many parameter trials. A strong historical result can therefore reflect search luck rather than a persistent signal. Standard summaries such as return, Sharpe ratio, and drawdown do not record how many candidates were tried, whether the selected rule survives out-of-sample validation, or whether the available history is long enough to support the result. This paper describes the MinervaScore, a post-selection robustness grade for trading strategies. The score combines four established validation quantities: Deflated Sharpe Ratio, Probability of Backtest Overfitting, Superior Predictive Ability, and Minimum Track Record Length, with a regime-stability diagnostic. These components are converted into signed margins from their admissibility thresholds, aggregated into a raw score, and then mapped to a 0-100 display. The display is tied to a binary Robustness Seal: scores of 80 or higher are shown only when all five gates pass. The calibration uses 359,062 production backtest records. The score is intended to rank statistical support, not to estimate the probability of future profit. In synthetic markets with known ground truth, the MinervaScore separates true signal from lucky backtest outcomes, with an AUROC of 0.989 at the headline difficulty. Its improvement over the GT-Score proxy and the gates-passed baseline is modest, and it remains close to the corrected DSR-alone baseline. In a pre-registered test on unseen real-market data, the score showed no significant forward relationship in a population with limited surviving edge (Spearman rho_s = 0.013, one-sided permutation p = 0.40). We therefore present the MinervaScore as an auditable validation and reporting layer, rather than as evidence of demonstrated real-market predictability.

Suggested Citation

  • Maria Laura Santoni & Vincent Jouanne & Matthew L. Scullin, 2026. "Equity Strategy Backtesting: Luck or Edge? The MinervaScore as a Statistical Robustness Grade," Papers 2608.23808, arXiv.org, revised Aug 2026.
  • Handle: RePEc:arx:papers:2608.23808
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/2608.23808
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Paolo Paruolo & Michaela Saisana & Andrea Saltelli, 2013. "Ratings and rankings: voodoo or science?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(3), pages 609-634, June.
    2. Whitney Newey & Kenneth West, 2014. "A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix," Applied Econometrics, Russian Academy of National Economy and Public Administration (RANEPA), vol. 33(1), pages 125-132.
    3. M. Saisana & A. Saltelli & S. Tarantola, 2005. "Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(2), pages 307-323, March.
    4. Blakeley B. McShane & David Gal & Andrew Gelman & Christian Robert & Jennifer L. Tackett, 2019. "Abandon Statistical Significance," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 235-245, March.
    5. Ronald L. Wasserstein & Nicole A. Lazar, 2016. "The ASA's Statement on p -Values: Context, Process, and Purpose," The American Statistician, Taylor & Francis Journals, vol. 70(2), pages 129-133, May.
    6. Dong Yin & Takeshi Miki & Vladislav Lesnichenko & Vasyl Gural, 2026. "Implementation Risk in Portfolio Backtesting: A Previously Unquantified Source of Error," Papers 2603.20319, arXiv.org.
    7. Joseph P. Romano & Michael Wolf, 2005. "Stepwise Multiple Testing as Formalized Data Snooping," Econometrica, Econometric Society, vol. 73(4), pages 1237-1282, July.
    8. Hansen, Peter Reinhard, 2005. "A Test for Superior Predictive Ability," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 365-380, October.
    9. Andrew W. Lo, 2002. "The Statistics of Sharpe Ratios," Financial Analysts Journal, Taylor & Francis Journals, vol. 58(4), pages 36-52, July.
    10. Alexander Pearson Sheppert, 2026. "The GT-Score: A Robust Objective Function for Reducing Overfitting in Data-Driven Trading Strategies," JRFM, MDPI, vol. 19(1), pages 1-12, January.
    11. Alexander Sheppert, 2026. "The GT-Score: A Robust Objective Function for Reducing Overfitting in Data-Driven Trading Strategies," Papers 2602.00080, arXiv.org.
    12. R. Cont, 2001. "Empirical properties of asset returns: stylized facts and statistical issues," Quantitative Finance, Taylor & Francis Journals, vol. 1(2), pages 223-236.
    13. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    14. Peter R. Hansen & Asger Lunde & James M. Nason, 2011. "The Model Confidence Set," Econometrica, Econometric Society, vol. 79(2), pages 453-497, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nikhil Devanathan & Alexandros E. Tzikas & Stephen P. Boyd, 2026. "Simple Dynamic Stock/Bond/Gold Portfolios," Papers 2609.07946, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Aoxin Zhang & Yuhan Cheng & Kwanting Leung, 2026. "Benchmarking Deep Time Series Models for Equity Portfolios," Papers 2606.09420, arXiv.org.
    2. Li, Jia & Patton, Andrew J., 2018. "Asymptotic inference about predictive accuracy using high frequency data," Journal of Econometrics, Elsevier, vol. 203(2), pages 223-240.
    3. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    4. Alexander Sheppert, 2026. "The GT-Score: A Robust Objective Function for Reducing Overfitting in Data-Driven Trading Strategies," Papers 2602.00080, arXiv.org.
    5. Owusu Junior, Peterson & Tiwari, Aviral Kumar & Tweneboah, George & Asafo-Adjei, Emmanuel, 2022. "GAS and GARCH based value-at-risk modeling of precious metals," Resources Policy, Elsevier, vol. 75(C).
    6. Mariano, Roberto S. & Preve, Daniel, 2012. "Statistical tests for multiple forecast comparison," Journal of Econometrics, Elsevier, vol. 169(1), pages 123-130.
    7. Lade, Gabriel & Lin, C.-Y. Cynthia & Smith, Aaron, 2014. "Policy Uncertainty under Market-Based Regulations: Evidence from the Renewable Fuel Standard," 2014 Annual Meeting, July 27-29, 2014, Minneapolis, Minnesota 170673, Agricultural and Applied Economics Association.
    8. Ding, Jing & Jiang, Lei & Liu, Xiaohui & Peng, Liang, 2023. "Nonparametric tests for market timing ability using daily mutual fund returns," Journal of Economic Dynamics and Control, Elsevier, vol. 150(C).
    9. Harris, Richard D.F. & Nguyen, Anh, 2013. "Long memory conditional volatility and asset allocation," International Journal of Forecasting, Elsevier, vol. 29(2), pages 258-273.
    10. David T. Frazier & Donald S. Poskitt, 2025. "Sequential Scoring Rule Evaluation for Forecast Method Selection," Papers 2505.09090, arXiv.org.
    11. Ahmed BenSaïda, 2021. "The Good and Bad Volatility: A New Class of Asymmetric Heteroskedastic Models," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 83(2), pages 540-570, April.
    12. Wang, Xueqing & Yuan, Ying & Qu, Yong, 2026. "Directional spillover effect and realized volatility forecasting: An extreme market regime switching model," Finance Research Letters, Elsevier, vol. 87(C).
    13. Hossein Hassani & Emmanuel Sirimal Silva, 2015. "A Kolmogorov-Smirnov Based Test for Comparing the Predictive Accuracy of Two Sets of Forecasts," Econometrics, MDPI, vol. 3(3), pages 1-20, August.
    14. Sumanjay Dutta & Shashi Jain, 2023. "Precision versus Shrinkage: A Comparative Analysis of Covariance Estimation Methods for Portfolio Allocation," Papers 2305.11298, arXiv.org.
    15. Degenhardt, Thomas & Auer, Benjamin R., 2018. "The “Sell in May” effect: A review and new empirical evidence," The North American Journal of Economics and Finance, Elsevier, vol. 43(C), pages 169-205.
    16. Hsu, Po-Hsuan & Hsu, Yu-Chin & Kuan, Chung-Ming, 2010. "Testing the predictive ability of technical analysis using a new stepwise test without data snooping bias," Journal of Empirical Finance, Elsevier, vol. 17(3), pages 471-484, June.
    17. Raffaella Giacomini & Barbara Rossi, 2013. "Forecasting in macroeconomics," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 17, pages 381-408, Edward Elgar Publishing.
    18. Xyngis, Georgios, 2017. "Business-cycle variation in macroeconomic uncertainty and the cross-section of expected returns: Evidence for scale-dependent risks," Journal of Empirical Finance, Elsevier, vol. 44(C), pages 43-65.
    19. Anghel, Dan Gabriel, 2021. "Data Snooping Bias in Tests of the Relative Performance of Multiple Forecasting Models," Journal of Banking & Finance, Elsevier, vol. 126(C).
    20. Hasanov, Akram Shavkatovich & Shaiban, Mohammed Sharaf & Al-Freedi, Ajab, 2020. "Forecasting volatility in the petroleum futures markets: A re-examination and extension," Energy Economics, Elsevier, vol. 86(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2608.23808. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.