Towards safer general aviation operations using a vision-based decision support system for weather threat avoidance

The commercial aviation sector has achieved significant advancements in safety owing to robust Air Traffic Management technologies and rigorous regulatory measures. In contrast, General Aviation (GA) operations present unique safety challenges that demand focused attention. This study proposes an innovative decision support system tailored for GA pilots to augment their situational awareness. Our approach leverages on-board camera data in conjunction with semantic weather descriptors to construct an uncertainty-aware neural network model. The model provides predictions with quantified uncertainties while handling multiple labels and categories across diverse weather conditions. To validate the effectiveness of our framework, extensive experiments were conducted utilizing a flight simulator as a data collection platform. The results demonstrate that our model showcased significant improvements over the multiple baselines. We also found that a cost-sensitive learning approach can provide more conservative predictions while yielding performance improvements. Ultimately, our decision support framework aims to complement existing weather data sources, such as Next Generation Weather Radar (NEXRAD) data and Meteorological Aerodrome Reports (METAR) from airports, without imposing the burden of mounting expensive and bulky on-board weather radar systems.