The medical licensing examination debate

National licensing examinations are typically large‐scale examinations taken early in a career or near the point of graduation, and, importantly, success is required to subsequently be able to practice. They are becoming increasingly popular as a method of quality assurance in the medical workforce, but debate about their contribution to patient safety and the improvement of healthcare outcomes continues. A systematic review of the national licensing examination literature demonstrates that there is disagreement between assessment experts about the strengths and challenges of licensing examinations. This is characterized by a trans‐Atlantic divide between the dominance of psychometric reliability assurance in North America and the wider interpretations of validity, to include consequences, in Europe. We conclude that the debate might benefit from refocusing to what a national licensing examination should assess: to achieve a balance between assessing a breadth of skills and the capacity for such skills in practice, and focusing less on reproducibility.