IDEAS home Printed from https://ideas.repec.org/a/eee/phsmap/v561y2021ics0378437120306178.html
   My bibliography  Save this article

Fitting a reversible Markov chain by maximum likelihood: Converting an awkwardly constrained optimization problem to an unconstrained one

Author

Listed:
  • MacDonald, Iain L.
  • Pienaar, Etienne A.D.

Abstract

We consider here the problem of fitting, by maximum likelihood, a discrete-time, finite-state–space Markov chain that is required to be reversible in time. The technique we use is to impose the detailed balance and other constraints by transforming the transition probabilities to a set of unconstrained ‘working parameters’, after which any general-purpose routine for unconstrained numerical optimization can be used to maximize the likelihood. The main advantages of this procedure are its simplicity and its use of standard, well-tested optimizers; very little computational expertise is needed, and all computations can easily be carried out in R on a standard machine. We provide several examples of applications, to simulated and other data, and give an example of the computation of standard errors and confidence intervals for parameters. We discuss also the variation of the problem in which it is required that the transition probabilities satisfy detailed balance with respect to a pre-specified stationary distribution, demonstrate that this case can be handled by a modification of our methods, but suggest two altogether different routes that may be preferable. A feature of our methods is the availability of excellent starting-values for the optimizations.

Suggested Citation

  • MacDonald, Iain L. & Pienaar, Etienne A.D., 2021. "Fitting a reversible Markov chain by maximum likelihood: Converting an awkwardly constrained optimization problem to an unconstrained one," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 561(C).
    • Handle: RePEc:eee:phsmap:v:561:y:2021:i:c:s0378437120306178
    DOI: 10.1016/j.physa.2020.125182
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0378437120306178
    Download Restriction: Full text for ScienceDirect subscribers only. Journal offers the option of making the article available online on Science direct for a fee of $3,000
    File URL: https://libkey.io/10.1016/j.physa.2020.125182?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Martin, Ian W. R. & Ross, Stephen A., 2019. "Notes on the yield curve," Journal of Financial Economics, Elsevier, vol. 134(3), pages 689-702.
    2. McCausland, William J., 2007. "Time reversibility of stationary regular finite-state Markov chains," Journal of Econometrics, Elsevier, vol. 136(1), pages 303-318, January.
    3. Annis, David H. & Kiessler, Peter C. & Lund, Robert & Steuber, Tara L., 2010. "Estimation in Reversible Markov Chains," The American Statistician, American Statistical Association, vol. 64(2), pages 116-120.
    4. Mahayaudin M. Mansor & David A. Green & Andrew V. Metcalfe, 2020. "Detecting Directionality in Time Series," The American Statistician, Taylor & Francis Journals, vol. 74(3), pages 258-266, July.
    5. MacDonald, Iain L., 2020. "A coarse-grained Markov chain is a hidden Markov model," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 541(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hansen, Stephen & McMahon, Michael & Tong, Matthew, 2019. "The long-run information effect of central bank communication," Journal of Monetary Economics, Elsevier, vol. 108(C), pages 185-202.
    2. Borovička, Jaroslav & Stachurski, John, 2021. "Stability of equilibrium asset pricing models: A necessary and sufficient condition," Journal of Economic Theory, Elsevier, vol. 193(C).
    3. Beare, Brendan K. & Seo, Juwon, 2014. "Time Irreversible Copula-Based Markov Models," Econometric Theory, Cambridge University Press, vol. 30(5), pages 923-960, October.
    4. Brendan K. Beare, 2010. "Copulas and Temporal Dependence," Econometrica, Econometric Society, vol. 78(1), pages 395-410, January.
    5. Jensen, Christian Skov & Lando, David & Pedersen, Lasse Heje, 2019. "Generalized recovery," Journal of Financial Economics, Elsevier, vol. 133(1), pages 154-174.
    6. McCausland, William J., 2007. "Time reversibility of stationary regular finite-state Markov chains," Journal of Econometrics, Elsevier, vol. 136(1), pages 303-318, January.
    7. Narayana R. Kocherlakota, 2023. "Infinite Debt Rollover in Stochastic Economies," Econometrica, Econometric Society, vol. 91(5), pages 1629-1658, September.
    8. Martin, Ian, 2018. "Options and the Gamma Knife," CEPR Discussion Papers 12883, C.E.P.R. Discussion Papers.
    9. Francesco Lancia & Alessia Russo & Tim Worrall, 2020. "Optimal Sustainable Intergenerational Insurance," Edinburgh School of Economics Discussion Paper Series 300, Edinburgh School of Economics, University of Edinburgh.
    10. Sergio Bacallado & Vijay Pande & Stefano Favaro & Lorenzo Trippa, 2016. "Bayesian regularization of the length of memory in reversible sequences," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(4), pages 933-946, September.
    11. J. Besag & D. Mondal, 2013. "Exact Goodness-of-Fit Tests for Markov Chains," Biometrics, The International Biometric Society, vol. 69(2), pages 488-496, June.
    12. Yong Chen & Jianmin Chen, 2011. "On the Imbedding Problem for Three-State Time Homogeneous Markov Chains with Coinciding Negative Eigenvalues," Journal of Theoretical Probability, Springer, vol. 24(4), pages 928-938, December.
    13. Davide Di Cecco, 2012. "Conditional exact tests for Markovianity and reversibility in multiple categorical sequences," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 21(1), pages 170-187, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:phsmap:v:561:y:2021:i:c:s0378437120306178. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/physica-a-statistical-mechpplications/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.