Generative AI extracts ecological meaning from the complex three dimensional shapes of bird bills

Data on the three dimensional shape of organismal morphology is becoming increasingly available, and forms part of a new revolution in high-throughput phenomics that promises to help understand ecological and evolutionary processes that influence phenotypes at unprecedented scales. However, in order to meet the potential of this revolution we need new data analysis tools to deal with the complexity and heterogeneity of large-scale phenotypic data such as 3D shapes. In this study we explore the potential of generative Artificial Intelligence to help organize and extract meaning from complex 3D data. Specifically, we train a deep representational learning method known as DeepSDF on a dataset of 3D scans of the bills of 2,020 bird species. The model is designed to learn a continuous vector representation of 3D shapes, along with a ’decoder’ function, that allows the transformation from this vector space to the original 3D morphological space. We find that approach successfully learns coherent representations: particular directions in latent space are associated with discernible morphological meaning (such as elongation, flattening, etc.). More importantly, learned latent vectors have ecological meaning as shown by their ability to predict the trophic niche of the bird each bill belongs to with a high degree of accuracy. Unlike existing 3D morphometric techniques, this method has very little requirements for human supervised tasks such as landmark placement, increasing it accessibility to labs with fewer labour resources. It has fewer strong assumptions than alternative dimension reduction techniques such as PCA. Once trained, 3D morphology predictions can be made from latent vectors very computationally cheaply. The trained model has been made publicly available and can be used by the community, including for finetuning on new data, representing an early step toward developing shared, reusable AI models for analyzing organismal morphology.Author summary: Scientists are now able to gather a wealth of information about the 3D shapes of organisms, which could revolutionize our understanding of how nature and evolution influence the forms of living creatures. Yet, to fully unlock this potential, we need new ways to handle and interpret such complex data. In this study, we’ve employed cutting-edge artificial intelligence (AI) to help sort out and make sense of intricate 3D shape data. To do this, we trained an advanced AI model on a database of 3D scans of bird beaks from over 2,000 different species. The AI was programmed to learn a simplified version of each 3D shape and to understand how to convert back and forth between this simplified form and the full 3D shape. We found that our AI model effectively learned to represent and interpret the forms of bird beaks. It was even able to predict the types of food a bird species might eat based on the simplified representation of its beak. This approach requires less human input and makes fewer assumptions than existing methods, providing a valuable new tool for analyzing animal morphology that complements existing methods and has many potentially promising downstream applications.