IDEAS home Printed from https://ideas.repec.org/a/bla/biomet/v77y2021i3p996-1010.html
   My bibliography  Save this article

A weak‐signal‐assisted procedure for variable selection and statistical inference with an informative subsample

Author

Listed:
  • Fang Fang
  • Jiwei Zhao
  • S. Ejaz Ahmed
  • Annie Qu

Abstract

This paper is motivated from an HIV‐1 drug resistance study where we encounter three analytical challenges: to analyze data with an informative subsample, to take into account the weak signals, and to detect important signals and also conduct statistical inference. We start with an initial estimation method, which adopts a penalized pairwise conditional likelihood approach for variable selection. This initial estimator incorporates the informative subsample issue. To accounting for the effect of weak signals, we use a key idea of partial ridge regression. We also propose a one‐step estimation method for each of the signal coefficients and then construct confidence intervals accordingly. We apply the proposed method to the Stanford HIV‐1 drug resistance study and compare the results with existing approaches. We also conduct comprehensive simulation studies to demonstrate the superior performance of our proposed method.

Suggested Citation

  • Fang Fang & Jiwei Zhao & S. Ejaz Ahmed & Annie Qu, 2021. "A weak‐signal‐assisted procedure for variable selection and statistical inference with an informative subsample," Biometrics, The International Biometric Society, vol. 77(3), pages 996-1010, September.
    • Handle: RePEc:bla:biomet:v:77:y:2021:i:3:p:996-1010
    DOI: 10.1111/biom.13346
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/biom.13346
    Download Restriction: no
    File URL: https://libkey.io/10.1111/biom.13346?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Bradley Efron, 2014. "Estimation and Accuracy After Model Selection," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(507), pages 991-1007, September.
    2. Xiaoli Gao & S. E. Ahmed & Yang Feng, 2017. "Post selection shrinkage estimation for high‐dimensional data analysis," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 33(2), pages 97-120, March.
    3. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    4. Cun-Hui Zhang & Stephanie S. Zhang, 2014. "Confidence intervals for low dimensional parameters in high dimensional linear models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 76(1), pages 217-242, January.
    5. Kung‐Yee Liang & Jing Qin, 2000. "Regression analysis under non‐standard situations: a pairwise pseudolikelihood approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 62(4), pages 773-786.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Liu, Yu & Zhuang, Xiaoyang, 2023. "Shrinkage estimation of semi-parametric spatial autoregressive panel data model with fixed effects," Statistics & Probability Letters, Elsevier, vol. 194(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alexandre Belloni & Victor Chernozhukov & Denis Chetverikov & Christian Hansen & Kengo Kato, 2018. "High-dimensional econometrics and regularized GMM," CeMMAP working papers CWP35/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    2. Chenchuan (Mark) Li & Ulrich K. Müller, 2021. "Linear regression with many controls of limited explanatory power," Quantitative Economics, Econometric Society, vol. 12(2), pages 405-442, May.
    3. Guo, Xu & Li, Runze & Liu, Jingyuan & Zeng, Mudong, 2023. "Statistical inference for linear mediation models with high-dimensional mediators and application to studying stock reaction to COVID-19 pandemic," Journal of Econometrics, Elsevier, vol. 235(1), pages 166-179.
    4. Toshio Honda, 2021. "The de-biased group Lasso estimation for varying coefficient models," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 73(1), pages 3-29, February.
    5. Hansen, Christian & Liao, Yuan, 2019. "The Factor-Lasso And K-Step Bootstrap Approach For Inference In High-Dimensional Economic Applications," Econometric Theory, Cambridge University Press, vol. 35(3), pages 465-509, June.
    6. Jingxuan Luo & Lili Yue & Gaorong Li, 2023. "Overview of High-Dimensional Measurement Error Regression Models," Mathematics, MDPI, vol. 11(14), pages 1-22, July.
    7. Wang, Yining & Wang, Jialei & Balakrishnan, Sivaraman & Singh, Aarti, 2019. "Rate optimal estimation and confidence intervals for high-dimensional regression with missing covariates," Journal of Multivariate Analysis, Elsevier, vol. 174(C).
    8. Panxu Yuan & Xiao Guo, 2022. "High-dimensional inference for linear model with correlated errors," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 85(1), pages 21-52, January.
    9. Agboola, Oluwagbenga David & Yu, Han, 2023. "Neighborhood-based cross fitting approach to treatment effects with high-dimensional data," Computational Statistics & Data Analysis, Elsevier, vol. 186(C).
    10. Victor Chernozhukov & Christian Hansen & Martin Spindler, 2015. "Valid Post-Selection and Post-Regularization Inference: An Elementary, General Approach," Annual Review of Economics, Annual Reviews, vol. 7(1), pages 649-688, August.
    11. Shu Lu & Yufeng Liu & Liang Yin & Kai Zhang, 2017. "Confidence intervals and regions for the lasso by using stochastic variational inequality techniques in optimization," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(2), pages 589-611, March.
    12. Victor Chernozhukov & Christian Hansen & Martin Spindler, 2015. "Post-Selection and Post-Regularization Inference in Linear Models with Many Controls and Instruments," American Economic Review, American Economic Association, vol. 105(5), pages 486-490, May.
    13. Yongli Zhang & Xiaotong Shen, 2015. "Adaptive Modeling Procedure Selection by Data Perturbation," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 33(4), pages 541-551, October.
    14. Peter Bühlmann & Domagoj Ćevid, 2020. "Deconfounding and Causal Regularisation for Stability and External Validity," International Statistical Review, International Statistical Institute, vol. 88(S1), pages 114-134, December.
    15. Lina Liao & Cheolwoo Park & Hosik Choi, 2019. "Penalized expectile regression: an alternative to penalized quantile regression," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 71(2), pages 409-438, April.
    16. Yoshimasa Uematsu & Takashi Yamagata, 2019. "Estimation of Weak Factor Models," DSSR Discussion Papers 96, Graduate School of Economics and Management, Tohoku University.
    17. Bai, Jushan & Liao, Yuan, 2017. "Inferences in panel data with interactive effects using large covariance matrices," Journal of Econometrics, Elsevier, vol. 200(1), pages 59-78.
    18. Liu, Yu & Zhuang, Xiaoyang, 2023. "Shrinkage estimation of semi-parametric spatial autoregressive panel data model with fixed effects," Statistics & Probability Letters, Elsevier, vol. 194(C).
    19. Zemin Zheng & Jinchi Lv & Wei Lin, 2021. "Nonsparse Learning with Latent Variables," Operations Research, INFORMS, vol. 69(1), pages 346-359, January.
    20. Max H. Farrell, 2013. "Robust Inference on Average Treatment Effects with Possibly More Covariates than Observations," Papers 1309.4686, arXiv.org, revised Feb 2018.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:biomet:v:77:y:2021:i:3:p:996-1010. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0006-341X .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.