Uniform post selection inference for LAD regression and other Z-estimation problems

We develop uniformly valid confidence regions for regression coefficients in a high-dimensional sparse median regression model with homoscedastic errors. Our methods are based on a moment equation that is immunized against non-regular estimation of the nuisance part of the median regression function by using Neyman’s orthogonalization. We establish that the resulting instrumental median regression estimator of a target regression coefficient is asymptotically normally distributed uniformly with respect to the underlying sparse model and is semiparametrically efficient. We also generalize our method to a general non-smooth Z-estimation framework with the number of target parameters p1 being possibly much larger than the sample size n. We extend Huber’s results on asymptotic normality to this setting, demonstrating uniform asymptotic normality of the proposed estimators over p1-dimensional rectangles, constructing simultaneous confidence bands on all of the p1 target parameters, and establishing asymptotic validity of the bands uniformly over underlying approximately sparse models.