Reliability of plastid and mitochondrial localisation prediction declines rapidly with the evolutionary distance to the training set increasing

Mitochondria and plastids import thousands of proteins. Their experimental localisation remains a frequent task, but can be resource-intensive and sometimes impossible. Hence, hundreds of studies make use of algorithms that predict a localisation based on a protein’s sequence. Their reliability across evolutionary diverse species is unknown. Here, we evaluate the performance of common algorithms (TargetP, Localizer and WoLFPSORT) for four photosynthetic eukaryotes (Arabidopsis thaliana, Zea mays, Physcomitrium patens, and Chlamydomonas reinhardtii) for which experimental plastid and mitochondrial proteome data is available, and 171 eukaryotes using orthology inferences. The match between predictions and experimental data ranges from 75% to as low as 2%. Results worsen as the evolutionary distance between training and query species increases, especially for plant mitochondria for which performance borders on random sampling. Specificity, sensitivity and precision analyses highlight cross-organelle errors and uncover the evolutionary divergence of organelles as the main driver of current performance issues. The results encourage to train the next generation of neural networks on an evolutionary more diverse set of organelle proteins for optimizing performance and reliability.Author summary: Recent advancements in genome sequencing and machine learning have been instrumental in solving numerous biological challenges, such as the prediction of the folded state of proteins from sequences alone. An intriguing cell biological challenge is tracking the localization of proteins within cells. For example, some nuclear-encoded proteins localize to mitochondria, while some are sorted to plastids. Experimentally tracking the localization of each protein across thousands of species is laborious. Instead, researchers use machine learning algorithms to predict where proteins are likely to be localized based on their sequence. How reliable are these predictions? We evaluated the reliability of prediction tools across more than a hundred plant species. We found that, as the evolutionary distance between the species used for training the algorithms and those used for testing increases, the accuracy of the predictions declines sharply. This perspective has allowed us to propose new strategies to improve these algorithms. We believe that training more distant plant genome sequences in combination with advances in artificial intelligence–and viewed through an evolutionary lens–will be crucial for developing localization prediction algorithms that are reliable across a wide range of species.