Development and validation of a scale to assess algorithmic literacy in the context of recommender systems (ALRS)

Algorithmic systems, particularly recommender systems, are central to online platforms yet remain largely opaque to users. Algorithmic literacy has therefore emerged as a critical competence to foster informed and autonomous engagement, especially among younger audiences. This study developed and validated a multidimensional scale—Algorithmic Literacy in the Context of Recommender Systems (ALRS)— focused on online audiovisual entertainment platforms. An initial pool of 51 items, derived from the literature, was refined through expert content validation, cognitive interviews, and lexical review, resulting in 38 items. The instrument was tested with 1564 Portuguese students aged 15–25. Reliability analyses showed strong internal consistency (α = 0.955; ω = 0.955; Spearman-Brown = 0.881). Exploratory factor analysis identified five dimensions, and confirmatory factor analysis on an independent sample supported a second-order structure with excellent fit indices (χ2/df = 1.55; CFI = 0.991; RMSEA = 0.040). Composite reliability and average variance extracted were also satisfactory. The final 29-item ALRS scale demonstrates robust psychometric properties across five dimensions—awareness, knowledge, assessment, reflection, and practical use—offering a valid and reliable instrument to assess algorithmic literacy in the context of recommender systems, with applications in education, research, and policy.