IDEAS home Printed from https://ideas.repec.org/a/eee/phsmap/v391y2012i4p1855-1864.html
   My bibliography  Save this article

Extractive summarization using complex networks and syntactic dependency

Author

Listed:
  • Amancio, Diego R.
  • Nunes, Maria G.V.
  • Oliveira, Osvaldo N.
  • Costa, Luciano da F.

Abstract

The realization that statistical physics methods can be applied to analyze written texts represented as complex networks has led to several developments in natural language processing, including automatic summarization and evaluation of machine translation. Most importantly, so far only a few metrics of complex networks have been used and therefore there is ample opportunity to enhance the statistics-based methods as new measures of network topology and dynamics are created. In this paper, we employ for the first time the metrics betweenness, vulnerability and diversity to analyze written texts in Brazilian Portuguese. Using strategies based on diversity metrics, a better performance in automatic summarization is achieved in comparison to previous work employing complex networks. With an optimized method the Rouge score (an automatic evaluation method used in summarization) was 0.5089, which is the best value ever achieved for an extractive summarizer with statistical methods based on complex networks for Brazilian Portuguese. Furthermore, the diversity metric can detect keywords with high precision, which is why we believe it is suitable to produce good summaries. It is also shown that incorporating linguistic knowledge through a syntactic parser does enhance the performance of the automatic summarizers, as expected, but the increase in the Rouge score is only minor. These results reinforce the suitability of complex network methods for improving automatic summarizers in particular, and treating text in general.

Suggested Citation

  • Amancio, Diego R. & Nunes, Maria G.V. & Oliveira, Osvaldo N. & Costa, Luciano da F., 2012. "Extractive summarization using complex networks and syntactic dependency," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 391(4), pages 1855-1864.
    • Handle: RePEc:eee:phsmap:v:391:y:2012:i:4:p:1855-1864
    DOI: 10.1016/j.physa.2011.10.015
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0378437111007953
    Download Restriction: Full text for ScienceDirect subscribers only. Journal offers the option of making the article available online on Science direct for a fee of $3,000
    File URL: https://libkey.io/10.1016/j.physa.2011.10.015?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Barabási, Albert-László & Albert, Réka & Jeong, Hawoong, 2000. "Scale-free characteristics of random networks: the topology of the world-wide web," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 281(1), pages 69-77.
    2. H. Bauke, 2007. "Parameter estimation for power-law distributions by maximum likelihood methods," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 58(2), pages 167-173, July.
    3. Viana, Matheus Palhares & Travençolo, Bruno A.N. & Tanck, Esther & Costa, Luciano da Fontoura, 2010. "Characterizing topological and dynamical properties of complex networks without border effects," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 389(8), pages 1771-1778.
    4. Amancio, D.R. & Nunes, M.G.V. & Oliveira, O.N. & Pardo, T.A.S. & Antiqueira, L. & da F. Costa, L., 2011. "Using metrics from complex networks to evaluate machine translation," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 390(1), pages 131-142.
    5. Antiqueira, L. & Nunes, M.G.V. & Oliveira Jr., O.N. & F. Costa, L. da, 2007. "Strong correlations between text quality and complex networks features," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 373(C), pages 811-820.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Martinčić-Ipšić, Sanda & Margan, Domagoj & Meštrović, Ana, 2016. "Multilayer network of language: A unified framework for structural analysis of linguistic subsystems," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 457(C), pages 117-128.
    2. Silva, Filipi N. & Amancio, Diego R. & Bardosova, Maria & Costa, Luciano da F. & Oliveira, Osvaldo N., 2016. "Using network science and text analytics to produce surveys in a scientific topic," Journal of Informetrics, Elsevier, vol. 10(2), pages 487-502.
    3. Amancio, Diego R. & Oliveira Jr., Osvaldo N. & Costa, Luciano da F., 2012. "Structure–semantics interplay in complex networks and its effects on the predictability of similarity in texts," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 391(18), pages 4406-4419.
    4. Dejian Yu & Wanru Wang & Shuai Zhang & Wenyu Zhang & Rongyu Liu, 2017. "Hybrid self-optimized clustering model based on citation links and textual features to detect research topics," PLOS ONE, Public Library of Science, vol. 12(10), pages 1-21, October.
    5. Tohalino, Jorge V. & Amancio, Diego R., 2018. "Extractive multi-document summarization using multilayer networks," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 503(C), pages 526-539.
    6. Diego Raphael Amancio, 2015. "Comparing the topological properties of real and artificially generated scientific manuscripts," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1763-1779, December.
    7. Diego R Amancio, 2015. "Probing the Topological Properties of Complex Networks Modeling Short Written Texts," PLOS ONE, Public Library of Science, vol. 10(2), pages 1-17, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. D. R. Amancio & M. G. V. Nunes & O. N. Oliveira & L. F. Costa, 2012. "Using complex networks concepts to assess approaches for citations in scientific papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 827-842, June.
    2. Wang, Yanhui & Bi, Lifeng & Lin, Shuai & Li, Man & Shi, Hao, 2017. "A complex network-based importance measure for mechatronics systems," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 466(C), pages 180-198.
    3. Amancio, Diego R. & Oliveira Jr., Osvaldo N. & Costa, Luciano da F., 2012. "Structure–semantics interplay in complex networks and its effects on the predictability of similarity in texts," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 391(18), pages 4406-4419.
    4. Liu, Yanyan & Li, Keping & Yan, Dongyang & Gu, Shuang, 2022. "A network-based CNN model to identify the hidden information in text data," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 590(C).
    5. Zhao, Star X. & Rousseau, Ronald & Ye, Fred Y., 2011. "h-Degree as a basic measure in weighted networks," Journal of Informetrics, Elsevier, vol. 5(4), pages 668-677.
    6. Ruiz Vargas, E. & Mitchell, D.G.V. & Greening, S.G. & Wahl, L.M., 2014. "Topology of whole-brain functional MRI networks: Improving the truncated scale-free model," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 405(C), pages 151-158.
    7. Giacomello, Giampiero & Picci, Lucio, 2003. "My scale or your meter? Evaluating methods of measuring the Internet," Information Economics and Policy, Elsevier, vol. 15(3), pages 363-383, September.
    8. Kwame Boamah‐Addo & Tomasz J. Kozubowski & Anna K. Panorska, 2023. "A discrete truncated Zipf distribution," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 77(2), pages 156-187, May.
    9. Ormerod, Paul & Roach, Andrew P, 2004. "The Medieval inquisition: scale-free networks and the suppression of heresy," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 339(3), pages 645-652.
    10. Castagna, Alina & Chentouf, Leila & Ernst, Ekkehard, 2017. "Economic vulnerabilities in Italy: A network analysis using similarities in sectoral employment," GLO Discussion Paper Series 50, Global Labor Organization (GLO).
    11. Pascal Billand & Christophe Bravard & Sudipta Sarangi, 2011. "Resources Flows Asymmetries in Strict Nash Networks with Partner Heterogeneity," Working Papers 1108, Groupe d'Analyse et de Théorie Economique Lyon St-Étienne (GATE Lyon St-Étienne), Université de Lyon.
    12. Stefan Rass, 2021. "Judging the quality of (fake) news on the internet," Mind & Society: Cognitive Studies in Economics and Social Sciences, Springer;Fondazione Rosselli, vol. 20(1), pages 129-133, June.
    13. Wei, Daijun & Deng, Xinyang & Zhang, Xiaoge & Deng, Yong & Mahadevan, Sankaran, 2013. "Identifying influential nodes in weighted networks based on evidence theory," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 392(10), pages 2564-2575.
    14. Pandey, Pradumn Kumar & Badarla, Venkataramana, 2018. "Reconstruction of network topology using status-time-series data," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 490(C), pages 573-583.
    15. Luis Garicano & Claire Lelarge & John Van Reenen, 2016. "Firm Size Distortions and the Productivity Distribution: Evidence from France," American Economic Review, American Economic Association, vol. 106(11), pages 3439-3479, November.
    16. Georgios Antonios Sarantitis & Theophilos Papadimitriou & Periklis Gogas, 2018. "A Network Analysis of the United Kingdom’s Consumer Price Index," Computational Economics, Springer;Society for Computational Economics, vol. 51(2), pages 173-193, February.
    17. Jiang, Jingchi & Zheng, Jichuan & Zhao, Chao & Su, Jia & Guan, Yi & Yu, Qiubin, 2016. "Clinical-decision support based on medical literature: A complex network approach," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 459(C), pages 42-54.
    18. Theo Frottier & Bertrand Georgeot & Olivier Giraud, 2022. "Harmonic structures of Beethoven quartets: a complex network approach," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 95(7), pages 1-8, July.
    19. Marc Barthélémy & Michele Campagna & Alessandro Chessa & Andrea De Montis & Alessandro Vespignani, 2005. "Emergent topological and dynamical properties of a real inter-municipal commuting network - perspectives for policy-making and planning," ERSA conference papers ersa05p607, European Regional Science Association.
    20. Tomson Ogwang, 2011. "Power laws in top wealth distributions: evidence from Canada," Empirical Economics, Springer, vol. 41(2), pages 473-486, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:phsmap:v:391:y:2012:i:4:p:1855-1864. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/physica-a-statistical-mechpplications/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.