IDEAS home Printed from https://ideas.repec.org/a/spr/jglopt/v68y2017i2d10.1007_s10898-016-0471-z.html
   My bibliography  Save this article

Accelerated parallel and distributed algorithm using limited internal memory for nonnegative matrix factorization

Author

Listed:
  • Duy Khuong Nguyen

    (Japan Advanced Institute of Science and Technology
    Vietnam National University)

  • Tu Bao Ho

    (Japan Advanced Institute of Science and Technology
    Vietnam National University)

Abstract

Nonnegative matrix factorization (NMF) is a powerful technique for dimension reduction, extracting latent factors and learning part-based representation. For large datasets, NMF performance depends on some major issues such as fast algorithms, fully parallel distributed feasibility and limited internal memory. This research designs a fast fully parallel and distributed algorithm using limited internal memory to reach high NMF performance for large datasets. Specially, we propose a flexible accelerated algorithm for NMF with all its $$L_1$$ L 1 $$L_2$$ L 2 regularized variants based on full decomposition, which is a combination of exact line search, greedy coordinate descent, and accelerated search. The proposed algorithm takes advantages of these algorithms to converges linearly at an over-bounded rate $$(1-\frac{\mu }{L})(1 - \frac{\mu }{rL})^{2r}$$ ( 1 - μ L ) ( 1 - μ r L ) 2 r in optimizing each factor matrix when fixing the other factor one in the sub-space of passive variables, where r is the number of latent components, and $$\mu $$ μ and L are bounded as $$\frac{1}{2} \le \mu \le L \le r$$ 1 2 ≤ μ ≤ L ≤ r . In addition, the algorithm can exploit the data sparseness to run on large datasets with limited internal memory of machines, which is is advanced compared to fast block coordinate descent methods and accelerated methods. Our experimental results are highly competitive with seven state-of-the-art methods about three significant aspects of convergence, optimality and average of the iteration numbers.

Suggested Citation

  • Duy Khuong Nguyen & Tu Bao Ho, 2017. "Accelerated parallel and distributed algorithm using limited internal memory for nonnegative matrix factorization," Journal of Global Optimization, Springer, vol. 68(2), pages 307-328, June.
  • Handle: RePEc:spr:jglopt:v:68:y:2017:i:2:d:10.1007_s10898-016-0471-z
    DOI: 10.1007/s10898-016-0471-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10898-016-0471-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s10898-016-0471-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. GILLIS, Nicolas & GLINEUR, François, 2011. "Accelerated multiplicative updates and hierarchical als algorithms for nonnegative matrix factorization," LIDAM Discussion Papers CORE 2011030, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    2. Daniel D. Lee & H. Sebastian Seung, 1999. "Learning the parts of objects by non-negative matrix factorization," Nature, Nature, vol. 401(6755), pages 788-791, October.
    3. NESTEROV, Yurii, 2012. "Efficiency of coordinate descent methods on huge-scale optimization problems," LIDAM Reprints CORE 2511, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    4. Berry, Michael W. & Browne, Murray & Langville, Amy N. & Pauca, V. Paul & Plemmons, Robert J., 2007. "Algorithms and applications for approximate nonnegative matrix factorization," Computational Statistics & Data Analysis, Elsevier, vol. 52(1), pages 155-173, September.
    5. Naiyang Guan & Lei Wei & Zhigang Luo & Dacheng Tao, 2013. "Limited-Memory Fast Gradient Descent Method for Graph Regularized Nonnegative Matrix Factorization," PLOS ONE, Public Library of Science, vol. 8(10), pages 1-10, October.
    6. Jingu Kim & Yunlong He & Haesun Park, 2014. "Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework," Journal of Global Optimization, Springer, vol. 58(2), pages 285-319, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Takehiro Sano & Tsuyoshi Migita & Norikazu Takahashi, 2022. "A novel update rule of HALS algorithm for nonnegative matrix factorization and Zangwill’s global convergence," Journal of Global Optimization, Springer, vol. 84(3), pages 755-781, November.
    2. Andrej Čopar & Blaž Zupan & Marinka Zitnik, 2019. "Fast optimization of non-negative matrix tri-factorization," PLOS ONE, Public Library of Science, vol. 14(6), pages 1-15, June.
    3. Rundong Du & Da Kuang & Barry Drake & Haesun Park, 2017. "DC-NMF: nonnegative matrix factorization based on divide-and-conquer for fast clustering and topic modeling," Journal of Global Optimization, Springer, vol. 68(4), pages 777-798, August.
    4. Jingu Kim & Yunlong He & Haesun Park, 2014. "Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework," Journal of Global Optimization, Springer, vol. 58(2), pages 285-319, February.
    5. Norikazu Takahashi & Ryota Hibi, 2014. "Global convergence of modified multiplicative updates for nonnegative matrix factorization," Computational Optimization and Applications, Springer, vol. 57(2), pages 417-440, March.
    6. M. Moghadam & K. Aminian & M. Asghari & M. Parnianpour, 2013. "How well do the muscular synergies extracted via non-negative matrix factorisation explain the variation of torque at shoulder joint?," Computer Methods in Biomechanics and Biomedical Engineering, Taylor & Francis Journals, vol. 16(3), pages 291-301.
    7. Gillis, Nicolas & Glineur, François & Tuyttens, Daniel & Vandaele, Arnaud, 2015. "Heuristics for exact nonnegative matrix factorization," LIDAM Discussion Papers CORE 2015006, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    8. Jianfei Cao & Han Yang & Jianshu Lv & Quanyuan Wu & Baolei Zhang, 2023. "Estimating Soil Salinity with Different Levels of Vegetation Cover by Using Hyperspectral and Non-Negative Matrix Factorization Algorithm," IJERPH, MDPI, vol. 20(4), pages 1-15, February.
    9. FUJIWARA Yoshi & INOUE Hiroyasu & YAMAGUCHI Takayuki & AOYAMA Hideaki & TANAKA Takuma & KIKUCHI Kentaro, 2021. "Money Flow Network Among Firms' Accounts in a Regional Bank of Japan," Discussion papers 21005, Research Institute of Economy, Trade and Industry (RIETI).
    10. Xiang Zhang & Naiyang Guan & Dacheng Tao & Xiaogang Qiu & Zhigang Luo, 2015. "Online Multi-Modal Robust Non-Negative Dictionary Learning for Visual Tracking," PLOS ONE, Public Library of Science, vol. 10(5), pages 1-17, May.
    11. Flavia Esposito, 2021. "A Review on Initialization Methods for Nonnegative Matrix Factorization: Towards Omics Data Experiments," Mathematics, MDPI, vol. 9(9), pages 1-17, April.
    12. Bastian Schaefermeier & Gerd Stumme & Tom Hanika, 2021. "Topic space trajectories," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5759-5795, July.
    13. Da Kuang & Sangwoon Yun & Haesun Park, 2015. "SymNMF: nonnegative low-rank approximation of a similarity matrix for graph clustering," Journal of Global Optimization, Springer, vol. 62(3), pages 545-574, July.
    14. Yoshi Fujiwara & Rubaiyat Islam, 2021. "Bitcoin's Crypto Flow Network," Papers 2106.11446, arXiv.org, revised Jul 2021.
    15. Yin Liu & Sam Davanloo Tajbakhsh, 2023. "Stochastic Composition Optimization of Functions Without Lipschitz Continuous Gradient," Journal of Optimization Theory and Applications, Springer, vol. 198(1), pages 239-289, July.
    16. Hiroyasu Abe & Hiroshi Yadohisa, 2019. "Orthogonal nonnegative matrix tri-factorization based on Tweedie distributions," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 13(4), pages 825-853, December.
    17. Guowei Yang & Lin Zhang & Minghua Wan, 2022. "Exponential Graph Regularized Non-Negative Low-Rank Factorization for Robust Latent Representation," Mathematics, MDPI, vol. 10(22), pages 1-20, November.
    18. Hiroyasu Abe & Hiroshi Yadohisa, 2017. "A non-negative matrix factorization model based on the zero-inflated Tweedie distribution," Computational Statistics, Springer, vol. 32(2), pages 475-499, June.
    19. Zhang, Lingsong & Lu, Shu & Marron, J.S., 2015. "Nested nonnegative cone analysis," Computational Statistics & Data Analysis, Elsevier, vol. 88(C), pages 100-110.
    20. Soodabeh Asadi & Janez Povh, 2021. "A Block Coordinate Descent-Based Projected Gradient Algorithm for Orthogonal Non-Negative Matrix Factorization," Mathematics, MDPI, vol. 9(5), pages 1-22, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:jglopt:v:68:y:2017:i:2:d:10.1007_s10898-016-0471-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.