SARSA in extended Kalman Filter for complex urban environments positioning

Nowadays, the Inertial Navigation System/Global Navigation Satellite System (INS/GNSS) integrated navigation system is widely used in many applications. The extended Kalman Filter (EKF) is a popular data fusion method for the INS/GNSS integrated navigation system. However, the process and measurement noise covariance matrices of the EKF cannot be modelled accurately due to varied scenes and complicated GNSS signal errors in urban environments, which undermines or deteriorates the EKF's performance. To mitigate noise covariance uncertainties' influence, this paper proposes an adaptive EKF algorithm named SARSA EKF, which enables the State-Action-Reward-State-Action (SARSA) method in EKF to realise the autonomous selection of the noise covariance matrices based on the Q-value. Meanwhile, a pruning algorithm is designed to remove inappropriate selections of noise covariance matrices and enhance the performance. The simulation and field test results indicate that the positioning accuracy of the SARSA EKF is better than the traditional EKF and the Q-learning EKF (QLEKF). The positioning accuracy's mean error of the SARSA EKF decreases by 34.32% and 25.95% compared with the traditional EKF and the QLEKF, respectively. And the positioning accuracy's standard deviation of the SARSA EKF decreases by 41.74% and 32.99% compared with the traditional EKF and the QLEKF, respectively.