IDEAS home Printed from https://ideas.repec.org/a/eee/phsmap/v611y2023ics0378437123000031.html
   My bibliography  Save this article

Visibility graph analysis of web server log files

Author

Listed:
  • Sulaimany, Sadegh
  • Mafakheri, Aso

Abstract

Web servers store every event in the form of logs, which contain the retrieved URL, client IP address, access time, HTTP status code, etc. There are several useful methods to analyze these log files, which are mainly based on sequential text mining techniques for applications like prefetching or driving critical information about system’s security. Finding ways to convert web server log files into graph structures may open new horizons in complex network analysis for investigation, comparison, and prediction. Since visibility graph has various successful usages for analyzing different time series data, web server log files as a kind of time-series data have the potential to be converted into visibility graph. In this research, we propose a novel method to convert web server log files into horizontal visibility graphs. Afterward, we demonstrate the result of the method on two popular datasets, NASA and Online Judge web server log files, and perform exploratory and visibility graph analysis techniques like centrality measures computation and community detection to show the promising future for the research. Moreover, we introduce a novel algorithm for a common application in web server log file analysis, web prefetching, based on a modified version of link prediction on the extracted visibility graph, and evaluate it based on AUC assessment and propose the next page to prefetch in each dataset. Finally, we propose several choices to extend the research in case of technical and practical aspects.

Suggested Citation

  • Sulaimany, Sadegh & Mafakheri, Aso, 2023. "Visibility graph analysis of web server log files," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 611(C).
    • Handle: RePEc:eee:phsmap:v:611:y:2023:i:c:s0378437123000031
    DOI: 10.1016/j.physa.2023.128448
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0378437123000031
    Download Restriction: Full text for ScienceDirect subscribers only. Journal offers the option of making the article available online on Science direct for a fee of $3,000
    File URL: https://libkey.io/10.1016/j.physa.2023.128448?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Yu, Xuan & Shi, Suixiang & Xu, Lingyu & Yu, Jie & Liu, Yaya, 2020. "Analyzing dynamic association of multivariate time series based on method of directed limited penetrable visibility graph," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 545(C).
    2. Bhaduri, Anirban & Bhaduri, Susmita & Ghosh, Dipak, 2017. "Visibility graph analysis of heart rate time series and bio-marker of congestive heart failure," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 482(C), pages 786-795.
    3. Lü, Linyuan & Zhou, Tao, 2011. "Link prediction in complex networks: A survey," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 390(6), pages 1150-1170.
    4. Tsiotas, Dimitrios & Charakopoulos, Avraam, 2018. "Visibility in the topology of complex networks," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 505(C), pages 280-292.
    5. Cao, Run-Hua & Deng, Zheng-Hong & Xu, Ji-Wei, 2022. "Analysis of precipitation characteristics in Shanghai based on the visibility graph algorithm," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 597(C).
    6. Zhu, Jia & Wei, Daijun, 2021. "Analysis of stock market based on visibility graph and structure entropy," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 576(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hu, Xiaohua & Niu, Min, 2023. "Horizontal visibility graphs mapped from multifractal trinomial measures," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 626(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Karimi, Fatemeh & Lotfi, Shahriar & Izadkhah, Habib, 2021. "Community-guided link prediction in multiplex networks," Journal of Informetrics, Elsevier, vol. 15(4).
    2. Chen, Ling-Jiao & Zhang, Zi-Ke & Liu, Jin-Hu & Gao, Jian & Zhou, Tao, 2017. "A vertex similarity index for better personalized recommendation," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 466(C), pages 607-615.
    3. Dong-Rui Chen & Chuang Liu & Yi-Cheng Zhang & Zi-Ke Zhang, 2019. "Predicting Financial Extremes Based on Weighted Visual Graph of Major Stock Indices," Complexity, Hindawi, vol. 2019, pages 1-17, October.
    4. Blagus, Neli & Šubelj, Lovro & Bajec, Marko, 2012. "Self-similar scaling of density in complex real-world networks," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 391(8), pages 2794-2802.
    5. Wei, Daijun & Deng, Xinyang & Zhang, Xiaoge & Deng, Yong & Mahadevan, Sankaran, 2013. "Identifying influential nodes in weighted networks based on evidence theory," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 392(10), pages 2564-2575.
    6. Xu, Hua & Wang, Minggang & Jiang, Shumin & Yang, Weiguo, 2020. "Carbon price forecasting with complex network and extreme learning machine," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 545(C).
    7. Andreas Spitz & Anna Gimmler & Thorsten Stoeck & Katharina Anna Zweig & Emőke-Ágnes Horvát, 2016. "Assessing Low-Intensity Relationships in Complex Networks," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-17, April.
    8. Liu, Chuang & Zhou, Wei-Xing, 2012. "Heterogeneity in initial resource configurations improves a network-based hybrid recommendation algorithm," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 391(22), pages 5704-5711.
    9. Shenshen Bai & Longjie Li & Jianjun Cheng & Shijin Xu & Xiaoyun Chen, 2018. "Predicting Missing Links Based on a New Triangle Structure," Complexity, Hindawi, vol. 2018, pages 1-11, December.
    10. Bowen Yan & Steve Gregory, 2013. "Identifying Communities and Key Vertices by Reconstructing Networks from Samples," PLOS ONE, Public Library of Science, vol. 8(4), pages 1-14, April.
    11. Zhang, Rong & Ashuri, Baabak & Shyr, Yu & Deng, Yong, 2018. "Forecasting Construction Cost Index based on visibility graph: A network approach," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 493(C), pages 239-252.
    12. Xia, Yongxiang & Pang, Wenbo & Zhang, Xuejun, 2021. "Mining relationships between performance of link prediction algorithms and network structure," Chaos, Solitons & Fractals, Elsevier, vol. 153(P2).
    13. Zhou, Ming-Yang & Xiong, Wen-Man & Wu, Xiang-Yang & Zhang, Yu-Xia & Liao, Hao, 2018. "Overlapping influence inspires the selection of multiple spreaders in complex networks," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 508(C), pages 76-83.
    14. Qiaoran Yang & Zhiliang Dong & Yichi Zhang & Man Li & Ziyi Liang & Chao Ding, 2021. "Who Will Establish New Trade Relations? Looking for Potential Relationship in International Nickel Trade," Sustainability, MDPI, vol. 13(21), pages 1-15, October.
    15. Weihua Lei & Luiz G. A. Alves & Luís A. Nunes Amaral, 2022. "Forecasting the evolution of fast-changing transportation networks using machine learning," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    16. Aslan, Serpil & Kaya, Buket & Kaya, Mehmet, 2019. "Predicting potential links by using strengthened projections in evolving bipartite networks," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 525(C), pages 998-1011.
    17. Leto Peel & Tiago P. Peixoto & Manlio De Domenico, 2022. "Statistical inference links data and theory in network science," Nature Communications, Nature, vol. 13(1), pages 1-15, December.
    18. Rafiee, Samira & Salavati, Chiman & Abdollahpouri, Alireza, 2020. "CNDP: Link prediction based on common neighbors degree penalization," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 539(C).
    19. Linyuan Lü & Yi-Cheng Zhang & Chi Ho Yeung & Tao Zhou, 2011. "Leaders in Social Networks, the Delicious Case," PLOS ONE, Public Library of Science, vol. 6(6), pages 1-9, June.
    20. Yin, Liang & Shi, Li-Chen & Zhao, Jun-Yan & Du, Song-Yang & Xie, Wen-Bo & Yuan, Fei & Chen, Duan-Bing, 2018. "Heterogeneous information network model for equipment-standard system," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 490(C), pages 935-943.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:phsmap:v:611:y:2023:i:c:s0378437123000031. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/physica-a-statistical-mechpplications/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.