Sub-sample Model Selection Procedures in Gets Modelling

When the DGP is nested in the model, PcGets delivers high performance selection across different (unknown) states of nature. One of its steps involves sub-sample post-selection assessment, and here we consider its properties and investigate its practical application. The simulation results show that conditional on retaining a variable, sub-sample information cannot discriminate between substantive and adventitious significance. The Monte Carlo experiments also reveal that the sub-sample selection method suggested by Hoover and Perez (1999) is dominated by procedures selecting only on full-sample evidence, when both approaches are evaluated at a given size. Nevertheless, although the sub-sample procedures do not result in a genuinely beneficial trade-off between size and power, they are particularly successful in controlling the size for selection problems that were previously seemed almost intractable.