IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v11y2023i10p2255-d1144783.html
   My bibliography  Save this article

Comparing Neural Style Transfer and Gradient-Based Algorithms in Brushstroke Rendering Tasks

Author

Listed:
  • Artur Karimov

    (Youth Research Institute, Saint Petersburg Electrotechnical University “LETI”, 197376 Saint Petersburg, Russia)

  • Ekaterina Kopets

    (Youth Research Institute, Saint Petersburg Electrotechnical University “LETI”, 197376 Saint Petersburg, Russia)

  • Tatiana Shpilevaya

    (Computer-Aided Design Department, Saint Petersburg Electrotechnical University “LETI”, 197376 Saint Petersburg, Russia)

  • Evgenii Katser

    (Computer-Aided Design Department, Saint Petersburg Electrotechnical University “LETI”, 197376 Saint Petersburg, Russia)

  • Sergey Leonov

    (Public Relations Department, Saint Petersburg Electrotechnical University “LETI”, 197376 Saint Petersburg, Russia)

  • Denis Butusov

    (Computer-Aided Design Department, Saint Petersburg Electrotechnical University “LETI”, 197376 Saint Petersburg, Russia)

Abstract

Non-photorealistic rendering (NPR) with explicit brushstroke representation is essential for both high-grade imitating of artistic paintings and generating commands for artistically skilled robots. Some algorithms for this purpose have been recently developed based on simple heuristics, e.g., using an image gradient for driving brushstroke orientation. The notable drawback of such algorithms is the impossibility of automatic learning to reproduce an individual artist’s style. In contrast, popular neural style transfer (NST) algorithms are aimed at this goal by their design. The question arises: how good is the performance of neural style transfer methods in comparison with the heuristic approaches? To answer this question, we develop a novel method for experimentally quantifying brushstroke rendering algorithms. This method is based on correlation analysis applied to histograms of six brushstroke parameters: length, orientation, straightness, number of neighboring brushstrokes (NBS-NB), number of brushstrokes with similar orientations in the neighborhood (NBS-SO), and orientation standard deviation in the neighborhood (OSD-NB). This method numerically captures similarities and differences in the distributions of brushstroke parameters and allows comparison of two NPR algorithms. We perform an investigation of the brushstrokes generated by the heuristic algorithm and the NST algorithm. The results imply that while the neural style transfer and the heuristic algorithms give rather different parameter histograms, their capabilities of mimicking individual artistic manner are limited comparably. A direct comparison of NBS-NB histograms of brushstrokes generated by these algorithms and of brushstrokes extracted from a real painting confirms this finding.

Suggested Citation

  • Artur Karimov & Ekaterina Kopets & Tatiana Shpilevaya & Evgenii Katser & Sergey Leonov & Denis Butusov, 2023. "Comparing Neural Style Transfer and Gradient-Based Algorithms in Brushstroke Rendering Tasks," Mathematics, MDPI, vol. 11(10), pages 1-30, May.
    • Handle: RePEc:gam:jmathe:v:11:y:2023:i:10:p:2255-:d:1144783
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/11/10/2255/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/11/10/2255/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Dowson, D. C. & Landau, B. V., 1982. "The Fréchet distance between multivariate normal distributions," Journal of Multivariate Analysis, Elsevier, vol. 12(3), pages 450-455, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Elham Yousefi & Luc Pronzato & Markus Hainy & Werner G. Müller & Henry P. Wynn, 2023. "Discrimination between Gaussian process models: active learning and static constructions," Statistical Papers, Springer, vol. 64(4), pages 1275-1304, August.
    2. Knott, Martin & Smith, Cyril, 2006. "Choosing joint distributions so that the variance of the sum is small," Journal of Multivariate Analysis, Elsevier, vol. 97(8), pages 1757-1765, September.
    3. Rippl, Thomas & Munk, Axel & Sturm, Anja, 2016. "Limit laws of the empirical Wasserstein distance: Gaussian distributions," Journal of Multivariate Analysis, Elsevier, vol. 151(C), pages 90-109.
    4. Zhongzhi Lawrence He, 2018. "Comparing Asset Pricing Models: Distance-based Metrics and Bayesian Interpretations," Papers 1803.01389, arXiv.org.
    5. Mordant, Gilles & Segers, Johan, 2022. "Measuring dependence between random vectors via optimal transport," Journal of Multivariate Analysis, Elsevier, vol. 189(C).
    6. Whiteley, Nick, 2021. "Dimension-free Wasserstein contraction of nonlinear filters," Stochastic Processes and their Applications, Elsevier, vol. 135(C), pages 31-50.
    7. Ledoit, Olivier & Wolf, Michael, 2021. "Shrinkage estimation of large covariance matrices: Keep it simple, statistician?," Journal of Multivariate Analysis, Elsevier, vol. 186(C).
    8. Puccetti, Giovanni & Rüschendorf, Ludger & Vanduffel, Steven, 2020. "On the computation of Wasserstein barycenters," Journal of Multivariate Analysis, Elsevier, vol. 176(C).
    9. Nabil Kahalé, 2019. "Efficient Simulation of High Dimensional Gaussian Vectors," Mathematics of Operations Research, INFORMS, vol. 44(1), pages 58-73, February.
    10. Xu, Ganggang & Zhu, Huirong & Lee, J. Jack, 2020. "Borrowing strength and borrowing index for Bayesian hierarchical models," Computational Statistics & Data Analysis, Elsevier, vol. 144(C).
    11. Abdulkabir Abdulraheem & Im Y. Jung, 2022. "A Comparative Study of Engraved-Digit Data Augmentation by Generative Adversarial Networks," Sustainability, MDPI, vol. 14(19), pages 1-14, September.
    12. Olivier Ledoit & Michael Wolf, 2019. "Shrinkage estimation of large covariance matrices: keep it simple, statistician?," ECON - Working Papers 327, Department of Economics - University of Zurich, revised Jun 2021.
    13. Zhongzhi Lawrence He, 2018. "Generalized Information Ratio," Papers 1803.01381, arXiv.org, revised Apr 2018.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:11:y:2023:i:10:p:2255-:d:1144783. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.