Heterogeneous trading strategies with adaptive fuzzy Actor-Critic reinforcement learning: A behavioral approach
AbstractThe present study addresses the learning mechanism of boundedly rational agents in the dynamic and noisy environment of financial markets. The main objective is the development of a system that "decodes" the knowledge-acquisition strategy and the decision-making process of technical analysts called "chartists". It advances the literature on heterogeneous learning in speculative markets by introducing a trading system wherein market environment and agent beliefs are represented by fuzzy inference rules. The resulting functionality leads to the derivation of the parameters of the fuzzy rules by means of adaptive training. In technical terms, it expands the literature that has utilized Actor-Critic reinforcement learning and fuzzy systems in agent-based applications, by presenting an adaptive fuzzy reinforcement learning approach that provides with accurate and prompt identification of market turning points and thus higher predictability. The purpose of this paper is to illustrate this concretely through a comparative investigation against other well-established models. The results indicate that with the inclusion of transaction costs, the profitability of the novel system in case of NASDAQ Composite, FTSE100 and NIKKEI255 indices is consistently superior to that of a Recurrent Neural Network, a Markov-switching model and a Buy and Hold strategy. Overall, the proposed system via the reinforcement learning mechanism, the fuzzy rule-based state space modeling and the adaptive action selection policy, leads to superior predictions upon the direction-of-change of the market.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.
Bibliographic InfoArticle provided by Elsevier in its journal Journal of Economic Dynamics and Control.
Volume (Year): 34 (2010)
Issue (Month): 6 (June)
Contact details of provider:
Web page: http://www.elsevier.com/locate/jedc
Agent-based modeling Technical trading Reinforcement learning Fuzzy inference Bounded rationality;
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Fernandez-Rodriguez, Fernando & Gonzalez-Martel, Christian & Sosvilla-Rivero, Simon, 2000. "On the profitability of technical trading rules based on artificial neural networks:: Evidence from the Madrid stock market," Economics Letters, Elsevier, vol. 69(1), pages 89-94, October.
- William Schwert, G., 2002.
"Stock volatility in the new millennium: how wacky is Nasdaq?,"
Journal of Monetary Economics,
Elsevier, vol. 49(1), pages 3-26, January.
- G. William Schwert, 2001. "Stock Volatility in the New Millennium: How Wacky Is Nasdaq?," NBER Working Papers 8436, National Bureau of Economic Research, Inc.
- Christoffersen, Peter F. & Diebold, Francis X., 2003.
"Financial asset returns, direction-of-change forecasting, and volatility dynamics,"
CFS Working Paper Series
2004/08, Center for Financial Studies (CFS).
- Peter F. Christoffersen & Francis X. Diebold, 2006. "Financial Asset Returns, Direction-of-Change Forecasting, and Volatility Dynamics," Management Science, INFORMS, vol. 52(8), pages 1273-1287, August.
- Peter F. Christoffersen & Francis X.Diebold, 2003. "Financial Asset Returns, Direction-of-Change Forecasting, and Volatility Dynamics," PIER Working Paper Archive 04-009, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
- Peter F. Christoffersen & Francis X. Diebold, 2003. "Financial Asset Returns, Direction-of-Change Forecasting, and Volatility Dynamics," NBER Working Papers 10009, National Bureau of Economic Research, Inc.
- Hommes, Cars H., 2006.
"Heterogeneous Agent Models in Economics and Finance,"
Handbook of Computational Economics,
in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 23, pages 1109-1186
- Cars H. Hommes, 2005. "Heterogeneous Agent Models in Economics and Finance," Tinbergen Institute Discussion Papers 05-056/1, Tinbergen Institute.
- La Porta, Rafael, et al, 1997.
" Good News for Value Stocks: Further Evidence on Market Efficiency,"
Journal of Finance,
American Finance Association, vol. 52(2), pages 859-74, June.
- Rafael La Porta & Josef Lakonishok & Andrei Shleifer & Robert Vishny, 1995. "Good News for Value Stocks: Further Evidence on Market Efficiency," NBER Working Papers 5311, National Bureau of Economic Research, Inc.
- Geert Bekaert & Guojun Wu, 1997.
"Asymmetric Volatility and Risk in Equity Markets,"
NBER Working Papers
6022, National Bureau of Economic Research, Inc.
- Robert J. Shiller, 2003.
"From Efficient Markets Theory to Behavioral Finance,"
Journal of Economic Perspectives,
American Economic Association, vol. 17(1), pages 83-104, Winter.
- Robert J. Shiller, 2002. "From Efficient Market Theory to Behavioral Finance," Cowles Foundation Discussion Papers 1385, Cowles Foundation for Research in Economics, Yale University.
- Cheol-Ho Park & Scott H. Irwin, 2007. "What Do We Know About The Profitability Of Technical Analysis?," Journal of Economic Surveys, Wiley Blackwell, vol. 21(4), pages 786-826, 09.
- Andrew Lo & Harry Mamaysky & Jiang Wang, 1999.
"Foundations of Technical Analysis: Computational Algorithms, Statistical Inference, and Empirical Implementation,"
Computing in Economics and Finance 1999
402, Society for Computational Economics.
- Andrew W. Lo & Harry Mamaysky & Jiang Wang, 2000. "Foundations of Technical Analysis: Computational Algorithms, Statistical Inference, and Empirical Implementation," Journal of Finance, American Finance Association, vol. 55(4), pages 1705-1770, 08.
- Andrew W. Lo & Harry Mamaysky & Jiang Wang, 2000. "Foundations of Technical Analysis: Computational Algorithms, Statistical Inference, and Empirical Implementation," NBER Working Papers 7613, National Bureau of Economic Research, Inc.
- Levich, Richard M. & Thomas, Lee III, 1993. "The significance of technical trading-rule profits in the foreign exchange market: a bootstrap approach," Journal of International Money and Finance, Elsevier, vol. 12(5), pages 451-474, October.
- Lawrence, Michael & O'Connor, Marcus, 1992. "Exploring judgemental forecasting," International Journal of Forecasting, Elsevier, vol. 8(1), pages 15-26, June.
- Fama, Eugene F, 1970. "Efficient Capital Markets: A Review of Theory and Empirical Work," Journal of Finance, American Finance Association, vol. 25(2), pages 383-417, May.
- repec:att:wimass:9621 is not listed on IDEAS
- Bollerslev, Tim, 1986.
"Generalized autoregressive conditional heteroskedasticity,"
Journal of Econometrics,
Elsevier, vol. 31(3), pages 307-327, April.
- Tim Bollerslev, 1986. "Generalized autoregressive conditional heteroskedasticity," EERI Research Paper Series EERI RP 1986/01, Economics and Econometrics Research Institute (EERI), Brussels.
- Po-Hsuan Hsu & Chung-Ming Kuan, 2005. "Reexamining the Profitability of Technical Analysis with Data Snooping Checks," Journal of Financial Econometrics, Society for Financial Econometrics, vol. 3(4), pages 606-628.
- Teo Jasic & Douglas Wood, 2004. "The profitability of daily stock market indices trades based on neural network predictions: case study for the S&P 500, the DAX, the TOPIX and the FTSE in the period 1965-1999," Applied Financial Economics, Taylor & Francis Journals, vol. 14(4), pages 285-297.
- Black, Fischer, 1986. " Noise," Journal of Finance, American Finance Association, vol. 41(3), pages 529-43, July.
- Franses,Philip Hans & Dijk,Dick van, 2000.
"Non-Linear Time Series Models in Empirical Finance,"
Cambridge University Press, number 9780521779654, December.
- Franses,Philip Hans & Dijk,Dick van, 2000. "Non-Linear Time Series Models in Empirical Finance," Cambridge Books, Cambridge University Press, number 9780521770415, December.
- Lawrence R. Glosten & Ravi Jagannathan & David E. Runkle, 1993.
"On the relation between the expected value and the volatility of the nominal excess return on stocks,"
157, Federal Reserve Bank of Minneapolis.
- Glosten, Lawrence R & Jagannathan, Ravi & Runkle, David E, 1993. " On the Relation between the Expected Value and the Volatility of the Nominal Excess Return on Stocks," Journal of Finance, American Finance Association, vol. 48(5), pages 1779-1801, December.
- Henriksson, Roy D & Merton, Robert C, 1981. "On Market Timing and Investment Performance. II. Statistical Procedures for Evaluating Forecasting Skills," The Journal of Business, University of Chicago Press, vol. 54(4), pages 513-33, October.
- Fama, Eugene F, 1991. " Efficient Capital Markets: II," Journal of Finance, American Finance Association, vol. 46(5), pages 1575-617, December.
- Tay, Nicholas S. P. & Linn, Scott C., 2001. "Fuzzy inductive reasoning, expectation formation and the behavior of security prices," Journal of Economic Dynamics and Control, Elsevier, vol. 25(3-4), pages 321-361, March.
- Paul R. Krugman, 1987. "Trigger Strategies and Price Dynamics in Equity and Foreign Exchange Markets," NBER Working Papers 2459, National Bureau of Economic Research, Inc.
- Brock, William A. & Hommes, Cars H., 1998.
"Heterogeneous beliefs and routes to chaos in a simple asset pricing model,"
Journal of Economic Dynamics and Control,
Elsevier, vol. 22(8-9), pages 1235-1274, August.
- Shleifer, Andrei & Summers, Lawrence H, 1990. "The Noise Trader Approach to Finance," Journal of Economic Perspectives, American Economic Association, vol. 4(2), pages 19-33, Spring.
- Gradojevic, Nikola & Gençay, Ramazan, 2013. "Fuzzy logic, trading uncertainty and technical trading," Journal of Banking & Finance, Elsevier, vol. 37(2), pages 578-586.
- Hommes, Cars, 2011.
"The heterogeneous expectations hypothesis: Some evidence from the lab,"
Journal of Economic Dynamics and Control,
Elsevier, vol. 35(1), pages 1-24, January.
- Hommes, C.H., 2010. "The Heterogeneous Expectations Hypothesis: Some Evidence from the Lab," CeNDEF Working Papers 10-06, Universiteit van Amsterdam, Center for Nonlinear Dynamics in Economics and Finance.
- Stelios Bekiros & Massimiliano Marcellino, 2011.
"The Multiscale Causal Dynamics of Foreign Exchange Markets,"
Economics Working Papers
ECO2011/23, European University Institute.
- Bekiros, Stelios & Marcellino, Massimiliano, 2013. "The multiscale causal dynamics of foreign exchange markets," Journal of International Money and Finance, Elsevier, vol. 33(C), pages 282-305.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei).
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If references are entirely missing, you can add them using this form.
If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.
Please note that corrections may take a couple of weeks to filter through the various RePEc services.