IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0217994.html
   My bibliography  Save this article

Fast optimization of non-negative matrix tri-factorization

Author

Listed:
  • Andrej Čopar
  • Blaž Zupan
  • Marinka Zitnik

Abstract

Non-negative matrix tri-factorization (NMTF) is a popular technique for learning low-dimensional feature representation of relational data. Currently, NMTF learns a representation of a dataset through an optimization procedure that typically uses multiplicative update rules. This procedure has had limited success, and its failure cases have not been well understood. We here perform an empirical study involving six large datasets comparing multiplicative update rules with three alternative optimization methods, including alternating least squares, projected gradients, and coordinate descent. We find that methods based on projected gradients and coordinate descent converge up to twenty-four times faster than multiplicative update rules. Furthermore, alternating least squares method can quickly train NMTF models on sparse datasets but often fails on dense datasets. Coordinate descent-based NMTF converges up to sixteen times faster compared to well-established methods.

Suggested Citation

  • Andrej Čopar & Blaž Zupan & Marinka Zitnik, 2019. "Fast optimization of non-negative matrix tri-factorization," PLOS ONE, Public Library of Science, vol. 14(6), pages 1-15, June.
  • Handle: RePEc:plo:pone00:0217994
    DOI: 10.1371/journal.pone.0217994
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0217994
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0217994&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0217994?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Birgin, Ernesto G. & Martínez, Jose Mario & Raydan, Marcos, 2014. "Spectral Projected Gradient Methods: Review and Perspectives," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 60(i03).
    2. GILLIS, Nicolas & GLINEUR, François, 2011. "Accelerated multiplicative updates and hierarchical als algorithms for nonnegative matrix factorization," LIDAM Discussion Papers CORE 2011030, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    3. Berry, Michael W. & Browne, Murray & Langville, Amy N. & Pauca, V. Paul & Plemmons, Robert J., 2007. "Algorithms and applications for approximate nonnegative matrix factorization," Computational Statistics & Data Analysis, Elsevier, vol. 52(1), pages 155-173, September.
    4. Jingu Kim & Yunlong He & Haesun Park, 2014. "Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework," Journal of Global Optimization, Springer, vol. 58(2), pages 285-319, February.
    5. Joachims, Thorsten, 1998. "Making large-scale SVM learning practical," Technical Reports 1998,28, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    6. NESTEROV, Yurii, 2012. "Efficiency of coordinate descent methods on huge-scale optimization problems," LIDAM Reprints CORE 2511, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    7. M. Merritt & Y. Zhang, 2005. "Interior-Point Gradient Method for Large-Scale Totally Nonnegative Least Squares Problems," Journal of Optimization Theory and Applications, Springer, vol. 126(1), pages 191-202, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Rok Hribar & Timotej Hrga & Gregor Papa & Gašper Petelin & Janez Povh & Nataša Pržulj & Vida Vukašinović, 2022. "Four algorithms to solve symmetric multi-type non-negative matrix tri-factorization problem," Journal of Global Optimization, Springer, vol. 82(2), pages 283-312, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Duy Khuong Nguyen & Tu Bao Ho, 2017. "Accelerated parallel and distributed algorithm using limited internal memory for nonnegative matrix factorization," Journal of Global Optimization, Springer, vol. 68(2), pages 307-328, June.
    2. Takehiro Sano & Tsuyoshi Migita & Norikazu Takahashi, 2022. "A novel update rule of HALS algorithm for nonnegative matrix factorization and Zangwill’s global convergence," Journal of Global Optimization, Springer, vol. 84(3), pages 755-781, November.
    3. Jingu Kim & Yunlong He & Haesun Park, 2014. "Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework," Journal of Global Optimization, Springer, vol. 58(2), pages 285-319, February.
    4. Gillis, Nicolas & Glineur, François & Tuyttens, Daniel & Vandaele, Arnaud, 2015. "Heuristics for exact nonnegative matrix factorization," LIDAM Discussion Papers CORE 2015006, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    5. V. S. Amaral & R. Andreani & E. G. Birgin & D. S. Marcondes & J. M. Martínez, 2022. "On complexity and convergence of high-order coordinate descent algorithms for smooth nonconvex box-constrained minimization," Journal of Global Optimization, Springer, vol. 84(3), pages 527-561, November.
    6. Rundong Du & Da Kuang & Barry Drake & Haesun Park, 2017. "DC-NMF: nonnegative matrix factorization based on divide-and-conquer for fast clustering and topic modeling," Journal of Global Optimization, Springer, vol. 68(4), pages 777-798, August.
    7. Norikazu Takahashi & Ryota Hibi, 2014. "Global convergence of modified multiplicative updates for nonnegative matrix factorization," Computational Optimization and Applications, Springer, vol. 57(2), pages 417-440, March.
    8. Luca Zanni, 2006. "An Improved Gradient Projection-based Decomposition Technique for Support Vector Machines," Computational Management Science, Springer, vol. 3(2), pages 131-145, April.
    9. M. Moghadam & K. Aminian & M. Asghari & M. Parnianpour, 2013. "How well do the muscular synergies extracted via non-negative matrix factorisation explain the variation of torque at shoulder joint?," Computer Methods in Biomechanics and Biomedical Engineering, Taylor & Francis Journals, vol. 16(3), pages 291-301.
    10. Roberto Andreani & Marcos Raydan, 2021. "Properties of the delayed weighted gradient method," Computational Optimization and Applications, Springer, vol. 78(1), pages 167-180, January.
    11. Rundong Du & Barry Drake & Haesun Park, 2019. "Hybrid clustering based on content and connection structure using joint nonnegative matrix factorization," Journal of Global Optimization, Springer, vol. 74(4), pages 861-877, August.
    12. Arnaud Vandaele & François Glineur & Nicolas Gillis, 2018. "Algorithms for positive semidefinite factorization," Computational Optimization and Applications, Springer, vol. 71(1), pages 193-219, September.
    13. Jianfei Cao & Han Yang & Jianshu Lv & Quanyuan Wu & Baolei Zhang, 2023. "Estimating Soil Salinity with Different Levels of Vegetation Cover by Using Hyperspectral and Non-Negative Matrix Factorization Algorithm," IJERPH, MDPI, vol. 20(4), pages 1-15, February.
    14. Filippozzi, Rafaela & Gonçalves, Douglas S. & Santos, Luiz-Rafael, 2023. "First-order methods for the convex hull membership problem," European Journal of Operational Research, Elsevier, vol. 306(1), pages 17-33.
    15. Xuefei Lu & Alessandro Rudi & Emanuele Borgonovo & Lorenzo Rosasco, 2020. "Faster Kriging: Facing High-Dimensional Simulators," Operations Research, INFORMS, vol. 68(1), pages 233-249, January.
    16. Peng Han & Xinyue Yang & Yifei Zhao & Xiangmin Guan & Shengjie Wang, 2022. "Quantitative Ground Risk Assessment for Urban Logistical Unmanned Aerial Vehicle (UAV) Based on Bayesian Network," Sustainability, MDPI, vol. 14(9), pages 1-13, May.
    17. TAYLOR, Adrien B. & HENDRICKX, Julien M. & François GLINEUR, 2016. "Exact worst-case performance of first-order methods for composite convex optimization," LIDAM Discussion Papers CORE 2016052, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    18. April R. Kriebel & Joshua D. Welch, 2022. "UINMF performs mosaic integration of single-cell multi-omic datasets using nonnegative matrix factorization," Nature Communications, Nature, vol. 13(1), pages 1-17, December.
    19. GILLIS, Nicolas & GLINEUR, François, 2011. "Accelerated multiplicative updates and hierarchical als algorithms for nonnegative matrix factorization," LIDAM Discussion Papers CORE 2011030, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    20. Shanika L Wickramasuriya & Berwin A Turlach & Rob J Hyndman, 2019. "Optimal Non-negative Forecast Reconciliation," Monash Econometrics and Business Statistics Working Papers 15/19, Monash University, Department of Econometrics and Business Statistics.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0217994. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.