IDEAS home Printed from https://ideas.repec.org/a/eee/phsmap/v611y2023ics0378437123000031.html
   My bibliography  Save this article

Visibility graph analysis of web server log files

Author

Listed:
  • Sulaimany, Sadegh
  • Mafakheri, Aso

Abstract

Web servers store every event in the form of logs, which contain the retrieved URL, client IP address, access time, HTTP status code, etc. There are several useful methods to analyze these log files, which are mainly based on sequential text mining techniques for applications like prefetching or driving critical information about system’s security. Finding ways to convert web server log files into graph structures may open new horizons in complex network analysis for investigation, comparison, and prediction. Since visibility graph has various successful usages for analyzing different time series data, web server log files as a kind of time-series data have the potential to be converted into visibility graph. In this research, we propose a novel method to convert web server log files into horizontal visibility graphs. Afterward, we demonstrate the result of the method on two popular datasets, NASA and Online Judge web server log files, and perform exploratory and visibility graph analysis techniques like centrality measures computation and community detection to show the promising future for the research. Moreover, we introduce a novel algorithm for a common application in web server log file analysis, web prefetching, based on a modified version of link prediction on the extracted visibility graph, and evaluate it based on AUC assessment and propose the next page to prefetch in each dataset. Finally, we propose several choices to extend the research in case of technical and practical aspects.

Suggested Citation

  • Sulaimany, Sadegh & Mafakheri, Aso, 2023. "Visibility graph analysis of web server log files," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 611(C).
    • Handle: RePEc:eee:phsmap:v:611:y:2023:i:c:s0378437123000031
    DOI: 10.1016/j.physa.2023.128448
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0378437123000031
    Download Restriction: Full text for ScienceDirect subscribers only. Journal offers the option of making the article available online on Science direct for a fee of $3,000
    File URL: https://libkey.io/10.1016/j.physa.2023.128448?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Yu, Xuan & Shi, Suixiang & Xu, Lingyu & Yu, Jie & Liu, Yaya, 2020. "Analyzing dynamic association of multivariate time series based on method of directed limited penetrable visibility graph," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 545(C).
    2. Bhaduri, Anirban & Bhaduri, Susmita & Ghosh, Dipak, 2017. "Visibility graph analysis of heart rate time series and bio-marker of congestive heart failure," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 482(C), pages 786-795.
    3. Lü, Linyuan & Zhou, Tao, 2011. "Link prediction in complex networks: A survey," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 390(6), pages 1150-1170.
    4. Tsiotas, Dimitrios & Charakopoulos, Avraam, 2018. "Visibility in the topology of complex networks," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 505(C), pages 280-292.
    5. Cao, Run-Hua & Deng, Zheng-Hong & Xu, Ji-Wei, 2022. "Analysis of precipitation characteristics in Shanghai based on the visibility graph algorithm," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 597(C).
    6. Zhu, Jia & Wei, Daijun, 2021. "Analysis of stock market based on visibility graph and structure entropy," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 576(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hu, Xiaohua & Niu, Min, 2023. "Horizontal visibility graphs mapped from multifractal trinomial measures," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 626(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chen, Ling-Jiao & Zhang, Zi-Ke & Liu, Jin-Hu & Gao, Jian & Zhou, Tao, 2017. "A vertex similarity index for better personalized recommendation," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 466(C), pages 607-615.
    2. Dong-Rui Chen & Chuang Liu & Yi-Cheng Zhang & Zi-Ke Zhang, 2019. "Predicting Financial Extremes Based on Weighted Visual Graph of Major Stock Indices," Complexity, Hindawi, vol. 2019, pages 1-17, October.
    3. Wei, Daijun & Deng, Xinyang & Zhang, Xiaoge & Deng, Yong & Mahadevan, Sankaran, 2013. "Identifying influential nodes in weighted networks based on evidence theory," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 392(10), pages 2564-2575.
    4. Weihua Lei & Luiz G. A. Alves & Luís A. Nunes Amaral, 2022. "Forecasting the evolution of fast-changing transportation networks using machine learning," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    5. Leto Peel & Tiago P. Peixoto & Manlio De Domenico, 2022. "Statistical inference links data and theory in network science," Nature Communications, Nature, vol. 13(1), pages 1-15, December.
    6. Rafiee, Samira & Salavati, Chiman & Abdollahpouri, Alireza, 2020. "CNDP: Link prediction based on common neighbors degree penalization," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 539(C).
    7. Linyuan Lü & Yi-Cheng Zhang & Chi Ho Yeung & Tao Zhou, 2011. "Leaders in Social Networks, the Delicious Case," PLOS ONE, Public Library of Science, vol. 6(6), pages 1-9, June.
    8. Yin, Liang & Shi, Li-Chen & Zhao, Jun-Yan & Du, Song-Yang & Xie, Wen-Bo & Yuan, Fei & Chen, Duan-Bing, 2018. "Heterogeneous information network model for equipment-standard system," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 490(C), pages 935-943.
    9. Wang, Zuxi & Wu, Yao & Li, Qingguang & Jin, Fengdong & Xiong, Wei, 2016. "Link prediction based on hyperbolic mapping with community structure for complex networks," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 450(C), pages 609-623.
    10. Kart, Ozge & Ulucay, Oguzhan & Bingol, Berkay & Isik, Zerrin, 2020. "A machine learning-based recommendation model for bipartite networks," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 553(C).
    11. Alireza Abbasi & Mahdi Jalili & Abolghasem Sadeghi-Niaraki, 2018. "Influence of network-based structural and power diversity on research performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 579-590, October.
    12. Lee, Yan-Li & Zhou, Tao, 2021. "Collaborative filtering approach to link prediction," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 578(C).
    13. Moradabadi, Behnaz & Meybodi, Mohammad Reza, 2016. "Link prediction based on temporal similarity metrics using continuous action set learning automata," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 460(C), pages 361-373.
    14. Jiang, Yawen & Jia, Caiyan & Yu, Jian, 2013. "An efficient community detection method based on rank centrality," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 392(9), pages 2182-2194.
    15. Park, Mingyu & Geum, Youngjung, 2022. "Two-stage technology opportunity discovery for firm-level decision making: GCN-based link-prediction approach," Technological Forecasting and Social Change, Elsevier, vol. 183(C).
    16. Yichi Zhang & Zhiliang Dong & Sen Liu & Peixiang Jiang & Cuizhi Zhang & Chao Ding, 2021. "Forecast of International Trade of Lithium Carbonate Products in Importing Countries and Small-Scale Exporting Countries," Sustainability, MDPI, vol. 13(3), pages 1-23, January.
    17. Yao, Can-Zhong & Lin, Ji-Nan & Zheng, Xu-Zhou & Liu, Xiao-Feng, 2015. "The study of RMB exchange rate complex networks based on fluctuation mode," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 436(C), pages 359-376.
    18. Liu, Zhenfeng & Feng, Jian & Uden, Lorna, 2023. "Technology opportunity analysis using hierarchical semantic networks and dual link prediction," Technovation, Elsevier, vol. 128(C).
    19. Shugang Li & Ziming Wang & Beiyan Zhang & Boyi Zhu & Zhifang Wen & Zhaoxu Yu, 2022. "The Research of “Products Rapidly Attracting Users” Based on the Fully Integrated Link Prediction Algorithm," Mathematics, MDPI, vol. 10(14), pages 1-19, July.
    20. Lee, Yan-Li & Zhou, Tao, 2017. "Fast asynchronous updating algorithms for k-shell indices," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 482(C), pages 524-531.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:phsmap:v:611:y:2023:i:c:s0378437123000031. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/physica-a-statistical-mechpplications/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.