IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2607.27544.html

Lucky or Good? Outcome Noise, Effective Sample Size, and the Attribution of Skill

Author

Listed:
  • Karl T. Ulrich

Abstract

When do outcome records carry enough signal to support reliable inferences about skill? When they do not, what should evaluators substitute? The framework answering the first question characterizes any decision domain with two parameters: the noise reflected in each outcome and the effective number of independent outcomes that are available over an observation window. When domains are positioned in a two-dimensional space of noise versus number of outcomes, those in which capital, prestige, and political power are routinely allocated on the basis of realized outcomes (e.g., mutual fund management, venture capital, executive performance) fall in the region where outcome records contain too little signal to support reliable individual-level inferences. Evaluating actors when outcome records are insufficient can be done by adopting the populationlevel empirical validation methods long used in medicine: has the actor adopted the practices that, at the population level, are associated with better outcomes?

Suggested Citation

  • Karl T. Ulrich, 2026. "Lucky or Good? Outcome Noise, Effective Sample Size, and the Attribution of Skill," Papers 2607.27544, arXiv.org.
    • Handle: RePEc:arx:papers:2607.27544
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/2607.27544
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Martijn Cremers, 2017. "Active Share and the Three Pillars of Active Management: Skill, Conviction, and Opportunity," Financial Analysts Journal, Taylor & Francis Journals, vol. 73(2), pages 61-79, April.
    2. Cochrane, John H., 2005. "The risk and return of venture capital," Journal of Financial Economics, Elsevier, vol. 75(1), pages 3-52, January.
    3. repec:bla:jfinan:v:55:y:2000:i:4:p:1655-1703 is not listed on IDEAS
    4. Martijn Cremers & Antti Petajisto, 2006. "How Active is Your Fund Manager? A New Measure That Predicts Performance," Yale School of Management Working Papers amz2370, Yale School of Management, revised 01 May 2009.
    5. Nicholas Bloom & John Van Reenen, 2007. "Measuring and Explaining Management Practices Across Firms and Countries," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(4), pages 1351-1408.
    6. Jerker Denrell & James G. March, 2001. "Adaptation as Information Restriction: The Hot Stove Effect," Organization Science, INFORMS, vol. 12(5), pages 523-538, October.
    7. Laurent Barras & Olivier Scaillet & Russ Wermers, 2010. "False Discoveries in Mutual Fund Performance: Measuring Luck in Estimated Alphas," Journal of Finance, American Finance Association, vol. 65(1), pages 179-216, February.
    8. Russ Wermers, 2000. "Mutual Fund Performance: An Empirical Decomposition into Stock‐Picking Talent, Style, Transactions Costs, and Expenses," Journal of Finance, American Finance Association, vol. 55(4), pages 1655-1695, August.
    9. Jerker Denrell, 2003. "Vicarious Learning, Undersampling of Failure, and the Myths of Management," Organization Science, INFORMS, vol. 14(3), pages 227-243, June.
    10. Marianne Bertrand & Sendhil Mullainathan, 2001. "Are CEOs Rewarded for Luck? The Ones Without Principals Are," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 116(3), pages 901-932.
    11. Dirk Jenter & Fadi Kanaan, 2015. "CEO Turnover and Relative Performance Evaluation," Journal of Finance, American Finance Association, vol. 70(5), pages 2155-2184, October.
    12. Jerker. Denrell & Christina. Fang & Daniel A. Levinthal, 2004. "From T-Mazes to Labyrinths: Learning from Model-Based Feedback," Management Science, INFORMS, vol. 50(10), pages 1366-1378, October.
    13. Loungani, Prakash, 2001. "How accurate are private sector forecasts? Cross-country evidence from consensus forecasts of output growth," International Journal of Forecasting, Elsevier, vol. 17(3), pages 419-432.
    14. Korteweg, Arthur & Sorensen, Morten, 2017. "Skill and luck in private equity performance," Journal of Financial Economics, Elsevier, vol. 124(3), pages 535-562.
    15. Marianne Bertrand & Antoinette Schoar, 2003. "Managing with Style: The Effect of Managers on Firm Policies," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(4), pages 1169-1208.
    16. Martijn Cremers & Antti Petajisto, 2006. "How Active is Your Fund Manager? A New Measure That Predicts Performance," Yale School of Management Working Papers amz2370, Yale School of Management, revised 01 May 2009.
    17. Robert H. Frank, 2016. "Success and Luck: Good Fortune and the Myth of Meritocracy," Economics Books, Princeton University Press, edition 1, number 10663, December.
    18. Eugene F. Fama & Kenneth R. French, 2010. "Luck versus Skill in the Cross‐Section of Mutual Fund Returns," Journal of Finance, American Finance Association, vol. 65(5), pages 1915-1947, October.
    19. K. J. Martijn Cremers & Antti Petajisto, 2009. "How Active Is Your Fund Manager? A New Measure That Predicts Performance," The Review of Financial Studies, Society for Financial Studies, vol. 22(9), pages 3329-3365, September.
    20. Josh Lerner & Antoinette Schoar & Wan Wongsunwai, 2007. "Smart Institutions, Foolish Choices: The Limited Partner Performance Puzzle," Journal of Finance, American Finance Association, vol. 62(2), pages 731-764, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. DeMiguel, Victor & Gil-Bazo, Javier & Nogales, Francisco J. & Santos, André A.P., 2023. "Machine learning and fund characteristics help to select mutual funds with positive alpha," Journal of Financial Economics, Elsevier, vol. 150(3).
    2. Clark Liu & Johan Sulaeman & Tao Shu & P Eric Yeung, 2023. "Life is Too Short? Bereaved Managers and Investment Decisions," Review of Finance, European Finance Association, vol. 27(4), pages 1373-1421.
    3. Jiang, Hao & Verardo, Michela, 2013. "Does herding behavior reveal skill? An analysis of mutual fund performance," LSE Research Online Documents on Economics 119034, London School of Economics and Political Science, LSE Library.
    4. Valentin Dimitrov & Prem C. Jain, 2026. "Yes, Virginia, there are superstar money managers," Review of Quantitative Finance and Accounting, Springer, vol. 66(2), pages 867-886, February.
    5. repec:osf:osfxxx:964ba_v1 is not listed on IDEAS
    6. Bai, John Jianqiu & Tang, Yuehua & Wan, Chi & Yüksel, H. Zafer, 2022. "Fund manager skill in an era of globalization: Offshore concentration and fund performance," Journal of Financial Economics, Elsevier, vol. 145(2), pages 18-40.
    7. Ekholm, Anders G., 2012. "Portfolio returns and manager activity: How to decompose tracking error into security selection and market timing," Journal of Empirical Finance, Elsevier, vol. 19(3), pages 349-358.
    8. Matallín-Sáez, Juan Carlos & Soler-Domínguez, Amparo & Tortosa-Ausina, Emili, 2016. "On the robustness of persistence in mutual fund performance," The North American Journal of Economics and Finance, Elsevier, vol. 36(C), pages 192-231.
    9. Habib, Michel & Johnsen, D. Bruce, 2015. "The quality-assuring role of mutual fund advisory fees," CEPR Discussion Papers 10438, Centre for Economic Policy Research.
    10. Gimeno, Ruth & Andreu, Laura & Sarto, José Luis, 2022. "Fund trading divergence and performance contribution," International Review of Financial Analysis, Elsevier, vol. 83(C).
    11. Livingston, Miles & Yao, Ping & Zhou, Lei, 2019. "The volatility of mutual fund performance," Journal of Economics and Business, Elsevier, vol. 104(C), pages 1-1.
    12. Victor DeMiguel & Javier Gil-Bazo & Francisco J. Nogales & André A. P. Santos, 2021. "Can machine learning help to select portfolios of mutual funds?," Economics Working Papers 1772, Department of Economics and Business, Universitat Pompeu Fabra.
    13. repec:bge:wpaper:1245 is not listed on IDEAS
    14. Habib, Michel A. & Johnsen, D. Bruce, 2016. "The quality-assuring role of mutual fund advisory fees," International Review of Law and Economics, Elsevier, vol. 46(C), pages 1-19.
    15. Huang, Haitao & Jiang, Lei & Leng, Xuan & Peng, Liang, 2023. "Bootstrap analysis of mutual fund performance," Journal of Econometrics, Elsevier, vol. 235(1), pages 239-255.
    16. Scott Bennett & David R Gallagher & Graham Harman & Geoffrey J Warren & Lihui Xi, 2016. "Alpha generation in portfolio management: Long-run Australian equity fund evidence," Australian Journal of Management, Australian School of Business, vol. 41(1), pages 107-140, February.
    17. Juan Carlos Matallín‐Sáez & Amparo Soler‐Domínguez & Diego Víctor de Mingo‐López, 2026. "Low‐Carbon Transition and Active Management: Evidence From Emerging Market Funds," Business Ethics, the Environment & Responsibility, John Wiley & Sons, Ltd., vol. 35(3), pages 1886-1906, July.
    18. Wu, Haoran & Gao, Zhiwei & Nie, Boyang & Zhao, Binru, 2025. "Can machines learn Chinese mutual funds?," Pacific-Basin Finance Journal, Elsevier, vol. 94(C).
    19. Carneiro, Livia Mendes & Eid Junior, William & Yoshinaga, Claudia Emiko, 2022. "The implications of passive investments for active fund management: International evidence," Global Finance Journal, Elsevier, vol. 53(C).
    20. Angelidis, Timotheos & Giamouridis, Daniel & Tessaromatis, Nikolaos, 2013. "Revisiting mutual fund performance evaluation," Journal of Banking & Finance, Elsevier, vol. 37(5), pages 1759-1776.
    21. Giovanni Walter Puopolo, 2026. "Mutual Fund Performance: A Review of the Literature," International Journal of Finance, CARI Journals Limited, vol. 11(1), pages 1-13.
    22. Suresh Nallareddy & Maria Ogneva, 2017. "Accrual quality, skill, and the cross-section of mutual fund returns," Review of Accounting Studies, Springer, vol. 22(2), pages 503-542, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2607.27544. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.