IDEAS home Printed from https://ideas.repec.org/a/bla/istatr/v92y2024i2p284-321.html
   My bibliography  Save this article

On the Inversion‐Free Newton's Method and Its Applications

Author

Listed:
  • Huy N. Chau
  • J. Lars Kirkby
  • Dang H. Nguyen
  • Duy Nguyen
  • Nhu N. Nguyen
  • Thai Nguyen

Abstract

In this paper, we survey the recent development of inversion‐free Newton's method, which directly avoids computing the inversion of Hessian, and demonstrate its applications in estimating parameters of models such as linear and logistic regression. A detailed review of existing methodology is provided, along with comparisons of various competing algorithms. We provide numerical examples that highlight some deficiencies of existing approaches, and demonstrate how the inversion‐free methods can improve performance. Motivated by recent works in literature, we provide a unified subsampling framework that can be combined with the inversion‐free Newton's method to estimate model parameters including those of linear and logistic regression. Numerical examples are provided for illustration.

Suggested Citation

  • Huy N. Chau & J. Lars Kirkby & Dang H. Nguyen & Duy Nguyen & Nhu N. Nguyen & Thai Nguyen, 2024. "On the Inversion‐Free Newton's Method and Its Applications," International Statistical Review, International Statistical Institute, vol. 92(2), pages 284-321, August.
    • Handle: RePEc:bla:istatr:v:92:y:2024:i:2:p:284-321
    DOI: 10.1111/insr.12563
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/insr.12563
    Download Restriction: no
    File URL: https://libkey.io/10.1111/insr.12563?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Yaqiong Yao & HaiYing Wang, 2019. "Optimal subsampling for softmax regression," Statistical Papers, Springer, vol. 60(2), pages 585-599, April.
    2. J. Lars Kirkby & Dang H. Nguyen & Duy Nguyen & Nhu N. Nguyen, 2022. "Inversion-free subsampling Newton’s method for large sample logistic regression," Statistical Papers, Springer, vol. 63(3), pages 943-963, June.
    3. Polyak, B.T., 2007. "Newton's method and its use in optimization," European Journal of Operational Research, Elsevier, vol. 181(3), pages 1086-1096, September.
    4. Haiying Wang & Yanyuan Ma, 2021. "Optimal subsampling for quantile regression in big data," Biometrika, Biometrika Trust, vol. 108(1), pages 99-112.
    5. HaiYing Wang & Min Yang & John Stufken, 2019. "Information-Based Optimal Subdata Selection for Big Data Linear Regression," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(525), pages 393-405, January.
    6. HaiYing Wang & Rong Zhu & Ping Ma, 2018. "Optimal Subsampling for Large Sample Logistic Regression," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(522), pages 829-844, April.
    7. Pelletier, Mariane, 1998. "On the almost sure asymptotic behaviour of stochastic algorithms," Stochastic Processes and their Applications, Elsevier, vol. 78(2), pages 217-244, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yue Chao & Lei Huang & Xuejun Ma & Jiajun Sun, 2024. "Optimal subsampling for modal regression in massive data," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 87(4), pages 379-409, May.
    2. Ziyang Wang & HaiYing Wang & Nalini Ravishanker, 2023. "Subsampling in Longitudinal Models," Methodology and Computing in Applied Probability, Springer, vol. 25(1), pages 1-29, March.
    3. Xiaohui Yuan & Yong Li & Xiaogang Dong & Tianqing Liu, 2022. "Optimal subsampling for composite quantile regression in big data," Statistical Papers, Springer, vol. 63(5), pages 1649-1676, October.
    4. Yan Song & Wenlin Dai, 2024. "Deterministic subsampling for logistic regression with massive data," Computational Statistics, Springer, vol. 39(2), pages 709-732, April.
    5. Min Ren & Shengli Zhao & Mingqiu Wang & Xinbei Zhu, 2024. "Robust optimal subsampling based on weighted asymmetric least squares," Statistical Papers, Springer, vol. 65(4), pages 2221-2251, June.
    6. Su, Miaomiao & Wang, Ruoyu & Wang, Qihua, 2022. "A two-stage optimal subsampling estimation for missing data problems with large-scale data," Computational Statistics & Data Analysis, Elsevier, vol. 173(C).
    7. Jun Yu & Jiaqi Liu & HaiYing Wang, 2023. "Information-based optimal subdata selection for non-linear models," Statistical Papers, Springer, vol. 64(4), pages 1069-1093, August.
    8. Shanshan Wang & Wei Cao & Xiaoxue Hu & Hanyu Zhong & Weixi Sun, 2025. "A Selective Overview of Quantile Regression for Large-Scale Data," Mathematics, MDPI, vol. 13(5), pages 1-30, March.
    9. Jun Yu & HaiYing Wang, 2022. "Subdata selection algorithm for linear model discrimination," Statistical Papers, Springer, vol. 63(6), pages 1883-1906, December.
    10. Xing Li & Yujing Shao & Lei Wang, 2024. "Optimal subsampling for $$L_p$$ L p -quantile regression via decorrelated score," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 33(4), pages 1084-1104, December.
    11. Tianzhen Wang & Haixiang Zhang, 2022. "Optimal subsampling for multiplicative regression with massive data," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 76(4), pages 418-449, November.
    12. J. Lars Kirkby & Dang H. Nguyen & Duy Nguyen & Nhu N. Nguyen, 2022. "Inversion-free subsampling Newton’s method for large sample logistic regression," Statistical Papers, Springer, vol. 63(3), pages 943-963, June.
    13. Amalan Mahendran & Helen Thompson & James M. McGree, 2023. "A model robust subsampling approach for Generalised Linear Models in big data settings," Statistical Papers, Springer, vol. 64(4), pages 1137-1157, August.
    14. Deng, Jiayi & Huang, Danyang & Ding, Yi & Zhu, Yingqiu & Jing, Bingyi & Zhang, Bo, 2024. "Subsampling spectral clustering for stochastic block models in large-scale networks," Computational Statistics & Data Analysis, Elsevier, vol. 189(C).
    15. Baolin Chen & Shanshan Song & Yong Zhou, 2024. "Estimation and testing of expectile regression with efficient subsampling for massive data," Statistical Papers, Springer, vol. 65(9), pages 5593-5613, December.
    16. Yuxin Sun & Wenjun Liu & Ye Tian, 2024. "Projection-Uniform Subsampling Methods for Big Data," Mathematics, MDPI, vol. 12(19), pages 1-16, September.
    17. Qian Yan & Hanyu Li & Chengmei Niu, 2023. "Optimal subsampling for functional quantile regression," Statistical Papers, Springer, vol. 64(6), pages 1943-1968, December.
    18. Hanji He & Jianfeng He & Liwei Zhang, 2025. "Imbalanced data sampling design based on grid boundary domain for big data," Computational Statistics, Springer, vol. 40(1), pages 27-64, January.
    19. Zhaolei Liu, 2025. "Optimal Subsampling for Upper Expectation Parametric Regression," Mathematics, MDPI, vol. 13(7), pages 1-24, March.
    20. Feifei Wang & Danyang Huang & Tianchen Gao & Shuyuan Wu & Hansheng Wang, 2022. "Sequential one‐step estimator by sub‐sampling for customer churn analysis with massive data sets," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 71(5), pages 1753-1786, November.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:istatr:v:92:y:2024:i:2:p:284-321. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/isiiinl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.