IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v11y2023i10p2320-d1148235.html
   My bibliography  Save this article

A Mathematical Investigation of Hallucination and Creativity in GPT Models

Author

Listed:
  • Minhyeok Lee

    (School of Electrical and Electronics Engineering, Chung-Ang University, Seoul 06974, Republic of Korea)

Abstract

In this paper, we present a comprehensive mathematical analysis of the hallucination phenomenon in generative pretrained transformer (GPT) models. We rigorously define and measure hallucination and creativity using concepts from probability theory and information theory. By introducing a parametric family of GPT models, we characterize the trade-off between hallucination and creativity and identify an optimal balance that maximizes model performance across various tasks. Our work offers a novel mathematical framework for understanding the origins and implications of hallucination in GPT models and paves the way for future research and development in the field of large language models (LLMs).

Suggested Citation

  • Minhyeok Lee, 2023. "A Mathematical Investigation of Hallucination and Creativity in GPT Models," Mathematics, MDPI, vol. 11(10), pages 1-17, May.
    • Handle: RePEc:gam:jmathe:v:11:y:2023:i:10:p:2320-:d:1148235
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/11/10/2320/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/11/10/2320/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Andrews, Donald W.K., 1992. "Generic Uniform Convergence," Econometric Theory, Cambridge University Press, vol. 8(2), pages 241-257, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Čížek, Pavel, 2008. "General Trimmed Estimation: Robust Approach To Nonlinear And Limited Dependent Variable Models," Econometric Theory, Cambridge University Press, vol. 24(6), pages 1500-1529, December.
    2. M. Hashem Pesaran & Yongcheol Shin, 2002. "Long-Run Structural Modelling," Econometric Reviews, Taylor & Francis Journals, vol. 21(1), pages 49-87.
    3. Donald W.K. Andrews & Werner Ploberger, 1994. "Testing for Serial Correlation Against an ARMA(1,1) Process," Cowles Foundation Discussion Papers 1077, Cowles Foundation for Research in Economics, Yale University.
    4. Sakata, Shinichi & White, Halbert, 2001. "S-estimation of nonlinear regression models with dependent and heterogeneous observations," Journal of Econometrics, Elsevier, vol. 103(1-2), pages 5-72, July.
    5. Donald W. K. Andrews & Xu Cheng, 2012. "Estimation and Inference With Weak, Semi‐Strong, and Strong Identification," Econometrica, Econometric Society, vol. 80(5), pages 2153-2211, September.
    6. Timo Dimitriadis & Xiaochun Liu & Julie Schnaitmann, 2020. "Encompassing Tests for Value at Risk and Expected Shortfall Multi-Step Forecasts based on Inference on the Boundary," Papers 2009.07341, arXiv.org.
    7. Zhu, Hongtu & Zhang, Heping, 2006. "Asymptotics for estimation and testing procedures under loss of identifiability," Journal of Multivariate Analysis, Elsevier, vol. 97(1), pages 19-45, January.
    8. Kristensen, Dennis & Rahbek, Anders, 2010. "Likelihood-based inference for cointegration with nonlinear error-correction," Journal of Econometrics, Elsevier, vol. 158(1), pages 78-94, September.
    9. Ketz, Philipp, 2019. "Testing overidentifying restrictions with a restricted parameter space," Economics Letters, Elsevier, vol. 185(C).
    10. Ghysels, Eric & Guay, Alain, 2004. "Testing For Structural Change In The Presence Of Auxiliary Models," Econometric Theory, Cambridge University Press, vol. 20(6), pages 1168-1202, December.
    11. Hansen, Bruce E. & Lee, Seojeong, 2019. "Asymptotic theory for clustered samples," Journal of Econometrics, Elsevier, vol. 210(2), pages 268-290.
    12. de Castro, Luciano & Galvao, Antonio F. & Kaplan, David M. & Liu, Xin, 2019. "Smoothed GMM for quantile models," Journal of Econometrics, Elsevier, vol. 213(1), pages 121-144.
    13. Xiaoxia Shi, 2015. "A nondegenerate Vuong test," Quantitative Economics, Econometric Society, vol. 6(1), pages 85-121, March.
    14. Ketz, Philipp, 2019. "On asymptotic size distortions in the random coefficients logit model," Journal of Econometrics, Elsevier, vol. 212(2), pages 413-432.
    15. Xiaohong Chen & Sokbae Lee & Myung Hwan Seo & Myunghyun Song, 2020. "Inference for parameters identified by conditional moment restrictions using a generalized Bierens maximum statistic," Papers 2008.11140, arXiv.org, revised Aug 2023.
    16. Cheng, Wei & Lee, Lung-fei, 2017. "Testing endogeneity of spatial and social networks," Regional Science and Urban Economics, Elsevier, vol. 64(C), pages 81-97.
    17. Hong, Yongmiao, 2001. "A test for volatility spillover with application to exchange rates," Journal of Econometrics, Elsevier, vol. 103(1-2), pages 183-224, July.
    18. Blevins, Jason R. & Kim, Minhae, 2024. "Nested Pseudo likelihood estimation of continuous-time dynamic discrete games," Journal of Econometrics, Elsevier, vol. 238(2).
    19. Lee, Sokbae & Liao, Yuan & Seo, Myung Hwan & Shin, Youngki, 2021. "Sparse HP filter: Finding kinks in the COVID-19 contact rate," Journal of Econometrics, Elsevier, vol. 220(1), pages 158-180.
    20. Feike C. Drost & Ramon Van Den Akker & Bas J. M. Werker, 2008. "Local asymptotic normality and efficient estimation for INAR(p) models," Journal of Time Series Analysis, Wiley Blackwell, vol. 29(5), pages 783-801, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:11:y:2023:i:10:p:2320-:d:1148235. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.