An English Multimodal Reading Assessment Framework Based on Eye-Tracking and Cross-Modal Integration

With the rise of visualization and mobile devices, English reading has become increasingly multimodal, yet assessments remain text-based and neglect systematic visual–text integration measurement. This study proposes an interpretable, low cognitive-load framework grounded in the eye-mind hypothesis and visual literacy theory. It introduces a three-dimensional model—information localization, crossmodal reasoning, context reconstruction—with weights from the Delphi method. A web-based eye-tracking plugin captures behavioral data in authentic contexts, and a multitask learning model enables automated scoring. Validation across three universities shows strong reliability (Cronbach's α = 0.87), validity (r = 0.64 between crossmodal reasoning and scores), and robust transferability across personal computers, tablets, smartphones, and virtual reality devices. The framework integrates cognitive theory and data-driven modeling, advancing multimodal assessment and supporting digital textbook design, online learning, and adaptive education.