IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v53y2009i7p2471-2484.html
   My bibliography  Save this article

Sharp quadratic majorization in one dimension

Author

Listed:
  • de Leeuw, Jan
  • Lange, Kenneth

Abstract

Majorization methods solve minimization problems by replacing a complicated problem by a sequence of simpler problems. Solving the sequence of simple optimization problems guarantees convergence to a solution of the complicated original problem. Convergence is guaranteed by requiring that the approximating functions majorize the original function at the current solution. The leading examples of majorization are the EM algorithm and the SMACOF algorithm used in Multidimensional Scaling. The simplest possible majorizing subproblems are quadratic, because minimizing a quadratic is easy to do. In this paper quadratic majorizations for real-valued functions of a real variable are analyzed, and the concept of sharp majorization is introduced and studied. Applications to logit, probit, and robust loss functions are discussed.

Suggested Citation

  • de Leeuw, Jan & Lange, Kenneth, 2009. "Sharp quadratic majorization in one dimension," Computational Statistics & Data Analysis, Elsevier, vol. 53(7), pages 2471-2484, May.
    • Handle: RePEc:eee:csdana:v:53:y:2009:i:7:p:2471-2484
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167-9473(09)00003-6
    Download Restriction: Full text for ScienceDirect subscribers only.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hunter D.R. & Lange K., 2004. "A Tutorial on MM Algorithms," The American Statistician, American Statistical Association, vol. 58, pages 30-37, February.
    2. Verboon, Peter & Heiser, Willem J., 1994. "Resistant lower rank approximation of matrices by iterative majorization," Computational Statistics & Data Analysis, Elsevier, vol. 18(4), pages 457-467, November.
    3. Groenen, P.J.F. & Giaquinto, P. & Kiers, H.A.L., 2003. "Weighted Majorization Algorithms for Weighted Least Squares Decomposition Models," Econometric Institute Research Papers EI 2003-09, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    4. Dankmar Böhning & Bruce Lindsay, 1988. "Monotonicity of quadratic-approximation algorithms," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 40(4), pages 641-663, December.
    5. de Leeuw, Jan, 2006. "Principal component analysis of binary data by iterated singular value decomposition," Computational Statistics & Data Analysis, Elsevier, vol. 50(1), pages 21-39, January.
    6. Heiser, Willem J., 1987. "Correspondence analysis with least absolute residuals," Computational Statistics & Data Analysis, Elsevier, vol. 5(4), pages 337-356, September.
    7. Werner Dinkelbach, 1967. "On Nonlinear Fractional Programming," Management Science, INFORMS, vol. 13(7), pages 492-498, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. John D. Rice & Brent A. Johnson & Robert L. Strawderman, 2022. "Screening for chronic diseases: optimizing lead time through balancing prescribed frequency and individual adherence," Lifetime Data Analysis: An International Journal Devoted to Statistical Methods and Applications for Time-to-Event Data, Springer, vol. 28(4), pages 605-636, October.
    2. Lloyd-Jones, Luke R. & Nguyen, Hien D. & McLachlan, Geoffrey J., 2018. "A globally convergent algorithm for lasso-penalized mixture of linear regression models," Computational Statistics & Data Analysis, Elsevier, vol. 119(C), pages 19-38.
    3. Kenneth Lange & Hua Zhou, 2022. "A Legacy of EM Algorithms," International Statistical Review, International Statistical Institute, vol. 90(S1), pages 52-66, December.
    4. Unkel, S. & Trendafilov, N.T., 2010. "A majorization algorithm for simultaneous parameter estimation in robust exploratory factor analysis," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3348-3358, December.
    5. Om Prakash Yadav & Shashwati Ray, 2021. "A novel method of preprocessing and modeling ECG signals with Lagrange–Chebyshev interpolating polynomials," International Journal of System Assurance Engineering and Management, Springer;The Society for Reliability, Engineering Quality and Operations Management (SREQOM),India, and Division of Operation and Maintenance, Lulea University of Technology, Sweden, vol. 12(3), pages 377-390, June.
    6. Kim, Nam-Hwui & Browne, Ryan P., 2021. "In the pursuit of sparseness: A new rank-preserving penalty for a finite mixture of factor analyzers," Computational Statistics & Data Analysis, Elsevier, vol. 160(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wang, Fa, 2017. "Maximum likelihood estimation and inference for high dimensional nonlinear factor models with application to factor-augmented regressions," MPRA Paper 93484, University Library of Munich, Germany, revised 19 May 2019.
    2. de Leeuw, Jan, 2006. "Principal component analysis of binary data by iterated singular value decomposition," Computational Statistics & Data Analysis, Elsevier, vol. 50(1), pages 21-39, January.
    3. Wang, Fa, 2022. "Maximum likelihood estimation and inference for high dimensional generalized factor models with application to factor-augmented regressions," Journal of Econometrics, Elsevier, vol. 229(1), pages 180-200.
    4. Utkarsh J. Dang & Michael P.B. Gallaugher & Ryan P. Browne & Paul D. McNicholas, 2023. "Model-Based Clustering and Classification Using Mixtures of Multivariate Skewed Power Exponential Distributions," Journal of Classification, Springer;The Classification Society, vol. 40(1), pages 145-167, April.
    5. Tian, Guo-Liang & Tang, Man-Lai & Liu, Chunling, 2012. "Accelerating the quadratic lower-bound algorithm via optimizing the shrinkage parameter," Computational Statistics & Data Analysis, Elsevier, vol. 56(2), pages 255-265.
    6. Landgraf, Andrew J. & Lee, Yoonkyung, 2020. "Dimensionality reduction for binary data through the projection of natural parameters," Journal of Multivariate Analysis, Elsevier, vol. 180(C).
    7. Henk Kiers, 1997. "Weighted least squares fitting using ordinary least squares algorithms," Psychometrika, Springer;The Psychometric Society, vol. 62(2), pages 251-266, June.
    8. Durante, Daniele & Canale, Antonio & Rigon, Tommaso, 2019. "A nested expectation–maximization algorithm for latent class models with covariates," Statistics & Probability Letters, Elsevier, vol. 146(C), pages 97-103.
    9. Ding, Jieli & Tian, Guo-Liang & Yuen, Kam Chuen, 2015. "A new MM algorithm for constrained estimation in the proportional hazards model," Computational Statistics & Data Analysis, Elsevier, vol. 84(C), pages 135-151.
    10. Tunjo Perić & Josip Matejaš & Zoran Babić, 2023. "Advantages, sensitivity and application efficiency of the new iterative method to solve multi-objective linear fractional programming problem," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 31(3), pages 751-767, September.
    11. Tien Mai & Arunesh Sinha, 2022. "Safe Delivery of Critical Services in Areas with Volatile Security Situation via a Stackelberg Game Approach," Papers 2204.11451, arXiv.org.
    12. Park, Chong Hyun & Lim, Heejong, 2021. "A parametric approach to integer linear fractional programming: Newton’s and Hybrid-Newton methods for an optimal road maintenance problem," European Journal of Operational Research, Elsevier, vol. 289(3), pages 1030-1039.
    13. Yong Xia & Longfei Wang & Xiaohui Wang, 2020. "Globally minimizing the sum of a convex–concave fraction and a convex function based on wave-curve bounds," Journal of Global Optimization, Springer, vol. 77(2), pages 301-318, June.
    14. H. Konno & K. Tsuchiya & R. Yamamoto, 2007. "Minimization of the Ratio of Functions Defined as Sums of the Absolute Values," Journal of Optimization Theory and Applications, Springer, vol. 135(3), pages 399-410, December.
    15. Henk Kiers, 1995. "Maximization of sums of quotients of quadratic forms and some generalizations," Psychometrika, Springer;The Psychometric Society, vol. 60(2), pages 221-245, June.
    16. Rasmus Lentz & Jean Marc Robin & Suphanit Piyapromdee, 2018. "On Worker and Firm Heterogeneity in Wages and Employment Mobility: Evidence from Danish Register Data," 2018 Meeting Papers 469, Society for Economic Dynamics.
    17. Luca Consolini & Marco Locatelli & Jiulin Wang & Yong Xia, 2020. "Efficient local search procedures for quadratic fractional programming problems," Computational Optimization and Applications, Springer, vol. 76(1), pages 201-232, May.
    18. Harald Dyckhoff & Katrin Allen, 1999. "Theoretische Begründung einer Effizienzanalyse mittels Data Envelopment Analysis (DEA)," Schmalenbach Journal of Business Research, Springer, vol. 51(5), pages 411-436, May.
    19. Smail Addoune & Karima Boufi & Ahmed Roubi, 2018. "Proximal Bundle Algorithms for Nonlinearly Constrained Convex Minimax Fractional Programs," Journal of Optimization Theory and Applications, Springer, vol. 179(1), pages 212-239, October.
    20. Feng Guo & Liguo Jiao, 2023. "A new scheme for approximating the weakly efficient solution set of vector rational optimization problems," Journal of Global Optimization, Springer, vol. 86(4), pages 905-930, August.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:53:y:2009:i:7:p:2471-2484. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.