From Data to Emotion: Accessible Emotion Recognition with Smartphone Sensors and Machine Learning

Emotions and moods significantly impact behavior and social interactions, making their prediction a valuable tool for enhancing communication and well-being. While previous systems, such as the Happimeter, utilized wearable devices to predict moods, accessibility remains limited due to the need for smartwatches or similar hardware. This study investigates emotion recognition using smartphone sensor data, offering a more ubiquitous and accessible platform. Moodacle, an iOS app, was developed to collect multimodal sensor data (accelerometer, gyroscope, GPS, microphone) and user-reported mood samples based on the Circumplex emotion model. Data from 45 participants over two weeks (1,338 sessions) was collected to train machine learning models to predict pleasance, activity, and combined mood states. Utilizing Boosted Trees demonstrated prediction results of 71% accuracy for pleasance, 43% for activity, and 39% for combined mood states across nine categories, outperforming random classifiers significantly. Our findings emphasize the potential of smartphones for scalable emotion recognition by combining internal phone sensor data such as accelerometer observations with environmental factors. Despite the limitations of an imbalanced dataset and reduced accuracy for certain mood states, this research highlights the feasibility of smartphone-based emotion recognition and provides a foundation for accessible, privacy-conscious mood tracking applications.