IonBench: A benchmark of optimisation strategies for mathematical models of ion channel currents

Ion channel models present many challenging optimisation problems. These include unidentifiable parameters, noisy data, unobserved states, and a combination of both fast and slow timescales. This can make it difficult to choose a suitable optimisation routine a priori. Nevertheless, many attempts have been made to design optimisation routines specifically for ion channel models, however, little work has been done to compare these optimisation approaches. We have developed ionBench, an open-source optimisation benchmarking framework, to evaluate and compare these approaches against a standard set of ion channel optimisation problems. We included implementations of thirty-four unique optimisation approaches that have been previously applied to ion channel models and evaluated them against the ionBench test suite, consisting of five parameter optimisation problems derived from the cardiac ion channel literature. Each optimisation approach was initiated from multiple starting parameters and tasked with reproducing a problem-specific simulated dataset. Through ionBench, we tracked and evaluated the performance of these optimisations, identifying the expected run time until a successful optimisation for each approach, which was used for comparisons. Finally, we used these results, in addition to other literature results, to identify a new efficient approach. Its use could reduce computation time by multiple orders of magnitude, while also improving the reliability of ion channel parameter optimisation.Author summary: To work with mathematical models of ion channels, model parameters must be configured to ensure the model can reproduce experimental data. This is achieved through a process called parameter optimisation. Many different methods and approaches have been developed to optimise parameters in ion channel models and they can vary in both speed (how fast they can optimise) and efficacy (how likely they are to produce parameters that can reproduce the data). However, even though this variability is known, a lack of standard problems has meant we still do not know the best way to optimise these models. We have compared thirty-four unique optimisation approaches that have been previously applied to ion channel models against a set of five optimisation problems derived from the literature. Finally, we used these results, in addition to other literature results, to identify a new efficient approach. Using this approach can lead to significant savings in computational resources and improvements in the reliability of parameter optimisation for ion channel models.