External Knowledge-Guided Tuning for Critical Error Detection in Machine Translation

With the advent of large language models (LLMs), significant progress has been made in improving the fluency of machine translation (MT). However, hallucination remains a persistent challenge to translation accuracy, making Critical Error Detection (CED) increasingly important. In this paper, we introduce a simple yet effective approach, termed external knowledge-guided tuning, for the CED task. We focus on sentence-level CED, formulated as a binary classification task that determines whether an MT output contains critical errors. Although the task is binary, the data consist of diverse error cases, including issues related to toxicity, safety, named entities, sentiment, and numerical information, which may manifest as hallucination, mistranslation, or deletion. Our approach restructures model inputs in a cloze-style format and incorporates auxiliary descriptions, casting CED within a masked language modeling framework. By integrating additional contextual signals, including demonstration examples and outputs from commercial systems, our method guides the model to acquire task-specific knowledge and compare alternative MT outputs. Experimental results demonstrate the effectiveness of our approach, achieving state-of-the-art (SOTA) performance on the English–Czech language pair and a second-place ranking on English–German. We further provide a comprehensive analysis of the aggregated effects of external knowledge and examine the contribution of each component within the proposed framework. Our proposed method enables the model to internalize task-relevant knowledge through parameter updates within a prompt-based formulation, providing a principled way to incorporate external knowledge into CED and enhancing the model’s ability to identify critical errors in practice.