LLM-as-a-Judge: Mastering Systematic Biases for Reliable Information Systems Adoption[LLM-juges: contrôle des biais systématiques pour une adoption fiable des Systèmes d'Information basés sur l'IA]

This article explores the LLM-as-a-Judge paradigm as a critical Information System (IS) whose reliability and neutrality are crucial factors for its organizational adoption. The study confirms the robustness of self-preference and highlights a massive position bias, indicating that the judgment is more sensitive to form and provenance than to intrinsic quality. This preference of LLM-judges for synthetic content reveals a risk of stylistic misalignment with human standards. The presence of these structural biases constitutes a major impediment to trust and adoption. Explicit and continuous bias measurement, along with the integration of diversified, blind, and human-calibrated evaluation pipelines, are essential to ensure reliable deployment.