The HESSIAN method: Highly efficient simulation smoothing, in a nutshell

I introduce the HESSIAN (highly efficient simulation smoothing in a nutshell) method for numerically efficient simulation smoothing in state space models with univariate states. Given a vector θ of parameters, the vector of states α=(α1,…,αn) is Gaussian and the observed vector y=(y1⊤,…,yn⊤)⊤ need not be. I describe a procedure to construct a close approximation q(α|θ,y) to the target density p(α|θ,y). It requires code to compute five derivatives of logp(yt|θ,αt) with respect to αt, t=1,…,n, and is not otherwise model specific. Since q(α|θ,y) is proper, fully normalised and simulable, it can be used as an importance density for importance sampling (IS) or as a proposal density for Markov chain Monte Carlo (MCMC). HESSIAN is an acronym but it also refers to the (sparse) Hessian matrix of logp(α|θ,y) with respect to α—the HESSIAN method is based on sparse matrix operations rather than the Kalman filter. I construct q(α|θ,y) and a related approximation q(θ,α|y) of p(θ,α|y) for two stochastic volatility models, two stochastic count models and a stochastic duration model. I illustrate their use for numerical approximation of likelihood function values and marginal likelihoods, using IS, and for posterior inference, using IS and MCMC. Compared with other simulation smoothing methods, the HESSIAN method is highly numerically efficient. In an IS application featuring a Student’s t stochastic volatility model and n=8851 daily log returns, the efficiency of IS for numerical approximation of the elements of the posterior mean E[θ|y] is between 80% and 100%.