On the latent state estimation of nonlinear population dynamics using Bayesian and non-Bayesian state-space models

Nonlinear state-space models have been increasingly applied to study population dynamics and data assimilation in environmental sciences. State-space models can account for process error and measurement error simultaneously to correct for the bias in the estimates of system state and model parameters. However, few studies have compared the performance of different nonlinear state-space models for reconstructing the state of population dynamics from noisy time series. This study compared the performance of the extended Kalman filter (EKF), unscented Kalman filter (UKF) and Bayesian nonlinear state-space models (BNSSM) through simulations. Synthetic population time series were generated using the theta logistic model with known parameters, and normally distributed process and measurement errors were introduced using the Monte Carlo simulations. At higher levels of nonlinearity, the UKF and BNSSM had lower root mean square error (RMSE) than the EKF. The BNSSM performed reliably across all levels of nonlinearity, whereas increased levels of nonlinearity resulted in higher RMSE of the EKF. The Metropolis–Hastings algorithm within the Gibbs algorithm was used to fit the theta logistic model to synthetic time series to estimate model parameters. The estimated posterior distribution of the parameter θ indicated that the 95% credible intervals included the true values of θ (=0.5 and 1.5), but did not include 1.0 and 0.0. Future studies need to incorporate the adaptive Metropolis algorithm to estimate unknown model parameters for broad applications of Bayesian nonlinear state-space models in ecological studies.