IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2507.16548.html
   My bibliography  Save this paper

Alternative Loss Function in Evaluation of Transformer Models

Author

Listed:
  • Jakub Micha'nk'ow
  • Pawe{l} Sakowski
  • Robert 'Slepaczuk

Abstract

The proper design and architecture of testing machine learning models, especially in their application to quantitative finance problems, is crucial. The most important aspect of this process is selecting an adequate loss function for training, validation, estimation purposes, and hyperparameter tuning. Therefore, in this research, through empirical experiments on equity and cryptocurrency assets, we apply the Mean Absolute Directional Loss (MADL) function, which is more adequate for optimizing forecast-generating models used in algorithmic investment strategies. The MADL function results are compared between Transformer and LSTM models, and we show that in almost every case, Transformer results are significantly better than those obtained with LSTM.

Suggested Citation

  • Jakub Micha'nk'ow & Pawe{l} Sakowski & Robert 'Slepaczuk, 2025. "Alternative Loss Function in Evaluation of Transformer Models," Papers 2507.16548, arXiv.org, revised Jul 2025.
    • Handle: RePEc:arx:papers:2507.16548
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2507.16548
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ru Zhang & Tong Cao, 2018. "Multi-factor Stock Selection Model Based on Adaboost," Business and Economic Research, Macrothink Institute, vol. 8(4), pages 119-129, December.
    2. Junming Yang & Yaoqi Li & Xuanyu Chen & Jiahang Cao & Kangkang Jiang, 2019. "Deep Learning for Stock Selection Based on High Frequency Price-Volume Data," Papers 1911.02502, arXiv.org.
    3. Jiayu Qiu & Bin Wang & Changjun Zhou, 2020. "Forecasting stock prices with long-short term memory neural network based on attention mechanism," PLOS ONE, Public Library of Science, vol. 15(1), pages 1-15, January.
    4. Ślepaczuk Robert & Sakowski Paweł & Zakrzewski Grzegorz, 2018. "Investment Strategies that Beat the Market. What Can We Squeeze from the Market?," Financial Internet Quarterly (formerly e-Finanse), Sciendo, vol. 14(4), pages 36-55, December.
    5. Ru Zhang & Chenyu Huang & Weijian Zhang & Shaozhen Chen, 2018. "Multi Factor Stock Selection Model Based on LSTM," International Journal of Economics and Finance, Canadian Center of Science and Education, vol. 10(8), pages 1-36, August.
    6. Zhen Zeng & Rachneet Kaur & Suchetha Siddagangappa & Saba Rahimi & Tucker Balch & Manuela Veloso, 2023. "Financial Time Series Forecasting using CNN and Transformer," Papers 2304.04912, arXiv.org.
    7. Grobys, Klaus & Ahmed, Shaker & Sapkota, Niranjan, 2020. "Technical trading rules in the cryptocurrency market," Finance Research Letters, Elsevier, vol. 32(C).
    8. Ru Zhang & Zi-ang Lin & Shaozhen Chen & Zhixuan Lin & Xingwei Liang, 2018. "Multi-factor Stock Selection Model Based on Kernel Support Vector Machine," Journal of Mathematics Research, Canadian Center of Science and Education, vol. 10(5), pages 9-18, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shuang Zhang & Xingdong Feng, 2022. "Distributed identification of heterogeneous treatment effects," Computational Statistics, Springer, vol. 37(1), pages 57-89, March.
    2. Jujie Wang & Zhenzhen Zhuang & Liu Feng, 2022. "Intelligent Optimization Based Multi-Factor Deep Learning Stock Selection Model and Quantitative Trading Strategy," Mathematics, MDPI, vol. 10(4), pages 1-19, February.
    3. Zeynep Cipiloglu Yildiz & Selim Baha Yildiz, 2022. "A portfolio construction framework using LSTM‐based stock markets forecasting," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 27(2), pages 2356-2366, April.
    4. Ganggang Guo & Yulei Rao & Feida Zhu & Fang Xu, 2020. "Innovative deep matching algorithm for stock portfolio selection using deep stock profiles," PLOS ONE, Public Library of Science, vol. 15(11), pages 1-31, November.
    5. Jakub Micha'nk'ow & Pawe{l} Sakowski & Robert 'Slepaczuk, 2023. "Mean Absolute Directional Loss as a New Loss Function for Machine Learning Problems in Algorithmic Investment Strategies," Papers 2309.10546, arXiv.org.
    6. Bouyaddou, Youssef & Jebabli, Ikram, 2025. "Integration of investor behavioral perspective and climate change in reinforcement learning for portfolio optimization," Research in International Business and Finance, Elsevier, vol. 73(PB).
    7. Wensheng Wang & Yuxi Liu, 2025. "A Novel Framework for Agricultural Futures Price Prediction With BERT‐Based Topic Identification and Sentiment Analysis," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 44(6), pages 1969-1992, September.
    8. Janusz Gajda & Rafał Walasek, 2020. "Fractional differentiation and its use in machine learning," Working Papers 2020-32, Faculty of Economic Sciences, University of Warsaw.
    9. Lorenzo Menculini & Andrea Marini & Massimiliano Proietti & Alberto Garinei & Alessio Bozza & Cecilia Moretti & Marcello Marconi, 2021. "Comparing Prophet and Deep Learning to ARIMA in Forecasting Wholesale Food Prices," Forecasting, MDPI, vol. 3(3), pages 1-19, September.
    10. Hoang, Lai & Vo, Duc Hong, 2024. "Google search and cross-section of cryptocurrency returns and trading activities," Journal of Behavioral and Experimental Finance, Elsevier, vol. 44(C).
    11. Ajithakumari Vijayappan Nair Biju & Ann Susan Thomas, 2023. "Uncertainties and ambivalence in the crypto market: an urgent need for a regional crypto regulation," SN Business & Economics, Springer, vol. 3(8), pages 1-21, August.
    12. Viktor Manahov, 2024. "The rapid growth of cryptocurrencies: How profitable is trading in digital money?," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 29(2), pages 2214-2229, April.
    13. Gustavo Iamin, 2024. "Are crypto-investors overconfident? The role of risk propensity and demographics. Evidence from Brazil and Portugal," Journal of Risk Finance, Emerald Group Publishing Limited, vol. 26(1), pages 147-173, November.
    14. Ergenç Cansu & Aktaş Rafet, 2025. "A Supervised Machine Learning in Financial Forecasting: Identifying Effective Models for the BIST100 Index," Review of Economic Perspectives, Sciendo, vol. 25(1), pages 66-90.
    15. Sapkota, Niranjan & Grobys, Klaus, 2021. "Asset market equilibria in cryptocurrency markets: Evidence from a study of privacy and non-privacy coins," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 74(C).
    16. Akash Doshi & Alexander Issa & Puneet Sachdeva & Sina Rafati & Somnath Rakshit, 2020. "Deep Stock Predictions," Papers 2006.04992, arXiv.org.
    17. Stefenon, Stefano Frizzo & Seman, Laio Oriel & Aquino, Luiza Scapinello & Coelho, Leandro dos Santos, 2023. "Wavelet-Seq2Seq-LSTM with attention for time series forecasting of level of dams in hydroelectric power plants," Energy, Elsevier, vol. 274(C).
    18. Renigier-Biłozor Małgorzata & Janowski Artur, 2024. "Human-Machine Synergy in Real Estate Similarity Concept," Real Estate Management and Valuation, Sciendo, vol. 32(2), pages 13-30.
    19. Deprez, Niek & Frömmel, Michael, 2024. "Are simple technical trading rules profitable in bitcoin markets?," International Review of Economics & Finance, Elsevier, vol. 93(PB), pages 858-874.
    20. Andreas Hackethal & Tobin Hanspal & Dominique M Lammer & Kevin Rink, 2022. "The Characteristics and Portfolio Behavior of Bitcoin Investors: Evidence from Indirect Cryptocurrency Investments [The investor in structured retail products: advice driven or gambling oriented]," Review of Finance, European Finance Association, vol. 26(4), pages 855-898.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2507.16548. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.