Settling the counterfactual debate: Is there a preferable counterfactual when estimating the returns to vocational qualifications?

Using information from the Longitudinal Education Outcomes (LEO) data set, it is now possible to compare the characteristics and estimates for learners with different qualifications encompassing both types of counterfactuals used in the extant literature: learners in possession of qualifications at the 'level-below' and learners enrolling in similar vocational qualifications but failing to achieve ('non-achievers'). In this analysis we adopt a Propensity Score Matching (PSM) method to assess whether it is possible to identify an 'optimal' counterfactual based on observable characteristics. In order to do that, we pool together the observations in the 'non-achievers' and 'level-below' counterfactual groups and compare the composition of the combined counterfactual group pre-match with the composition of the same group post-match. If neither group is preferable, then the breakdown of the matched counterfactual group should be in proportion to the relative sample sizes pre-match. If this is not the case, and one group is relatively over-represented post-matching, then there is a preference for that particular control group in terms of observable characteristics only. We find that, for both males and females, the non-achiever group is generally overrepresented for qualifications at Level 2 and above. That is, non-achievers are generally closer in their observable characteristics to the achievers, than are individuals who only complete the qualification at the level below. Finally, earnings differentials estimated using the 'non-achievers' group tend to be smaller than differentials estimated using the 'level-below' group, and this is especially true for male individuals.