IDEAS home Printed from https://ideas.repec.org/a/igg/jswis0/v19y2023i1p1-20.html
   My bibliography  Save this article

Query-Guided Refinement and Dynamic Spans Network for Video Highlight Detection and Temporal Grounding in Online Information Systems

Author

Listed:
  • Yifang Xu

    (School of Electronic Science and Engineering, Nanjing University, China)

  • Yunzhuo Sun

    (School of Physics and Electronics, Hubei Normal University, China)

  • Zien Xie

    (School of Electronic Science and Engineering, Nanjing University, China)

  • Benxiang Zhai

    (School of Electronic Science and Engineering, Nanjing University, China)

  • Youyao Jia

    (Gosuncn Chuanglian Technology Co., Ltd., Guangzhou, China)

  • Sidan Du

    (School of Electronic Science and Engineering, Nanjing University, China)

Abstract

With the surge in online video content, finding highlights and key video segments have garnered widespread attention. Given a textual query, video highlight detection (HD) and temporal grounding (TG) aim to predict frame-wise saliency scores from a video while concurrently locating all relevant spans. Despite recent progress in DETR-based works, these methods crudely fuse different inputs in the encoder, which limits effective cross-modal interaction. To solve this challenge, the authors design QD-Net (query-guided refinement and dynamic spans network) tailored for HD&TG. Specifically, they propose a query-guided refinement module to decouple the feature encoding from the interaction process. Furthermore, they present a dynamic span decoder that leverages learnable 2D spans as decoder queries, which accelerates training convergence for TG. On QVHighlights dataset, the proposed QD-Net achieves 61.87 HD-HIT@1 and 61.88 TG-mAP@0.5, yielding a significant improvement of +1.88 and +8.05, respectively, compared to the state-of-the-art method.

Suggested Citation

  • Yifang Xu & Yunzhuo Sun & Zien Xie & Benxiang Zhai & Youyao Jia & Sidan Du, 2023. "Query-Guided Refinement and Dynamic Spans Network for Video Highlight Detection and Temporal Grounding in Online Information Systems," International Journal on Semantic Web and Information Systems (IJSWIS), IGI Global, vol. 19(1), pages 1-20, January.
    • Handle: RePEc:igg:jswis0:v:19:y:2023:i:1:p:1-20
    as

    Download full text from publisher

    File URL: https://services.igi-global.com/resolvedoi/resolve.aspx?doi=10.4018/IJSWIS.332768
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Lee, Jung Hoon & Hancock, Marguerite Gong & Hu, Mei-Chih, 2014. "Towards an effective framework for building smart cities: Lessons from Seoul and San Francisco," Technological Forecasting and Social Change, Elsevier, vol. 89(C), pages 80-99.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sabina Baraniewicz-Kotasińska, 2022. "The Scandinavian Third Way as a Proposal for Sustainable Smart City Development—A Case Study of Aarhus City," Sustainability, MDPI, vol. 14(6), pages 1-24, March.
    2. Łukasz Brzeziński & Magdalena Krystyna Wyrwicka, 2022. "Fundamental Directions of the Development of the Smart Cities Concept and Solutions in Poland," Energies, MDPI, vol. 15(21), pages 1-52, November.
    3. Kajikawa, Yuya & Mejia, Cristian & Wu, Mengjia & Zhang, Yi, 2022. "Academic landscape of Technological Forecasting and Social Change through citation network and topic analyses," Technological Forecasting and Social Change, Elsevier, vol. 182(C).
    4. Mona Treude, 2021. "Sustainable Smart City—Opening a Black Box," Sustainability, MDPI, vol. 13(2), pages 1-15, January.
    5. Niusha Esmaeilpoorarabi & Tan Yigitcanlar & Mirko Guaralda, 2016. "Place quality and urban competitiveness symbiosis? A position paper," International Journal of Knowledge-Based Development, Inderscience Enterprises Ltd, vol. 7(1), pages 4-21.
    6. Abbate, Tindara & Cesaroni, Fabrizio & Cinici, Maria Cristina & Villari, Massimo, 2019. "Business models for developing smart cities. A fuzzy set qualitative comparative analysis of an IoT platform," Technological Forecasting and Social Change, Elsevier, vol. 142(C), pages 183-193.
    7. Małgorzata Baran & Monika Kłos & Monika Chodorek & Karolina Marchlewska-Patyk, 2022. "The Resilient Smart City Model–Proposal for Polish Cities," Energies, MDPI, vol. 15(5), pages 1-23, March.
    8. Seung-Min Jung & Sungwoo Park & Seung-Won Jung & Eenjun Hwang, 2020. "Monthly Electric Load Forecasting Using Transfer Learning for Smart Cities," Sustainability, MDPI, vol. 12(16), pages 1-20, August.
    9. Nilssen, Maja, 2019. "To the smart city and beyond? Developing a typology of smart urban innovation," Technological Forecasting and Social Change, Elsevier, vol. 142(C), pages 98-104.
    10. Francesco Schiavone & Francesco Paolo Appio & Luca Mora & Marcello Risitano, 2020. "The strategic, organizational, and entrepreneurial evolution of smart cities," International Entrepreneurship and Management Journal, Springer, vol. 16(4), pages 1155-1165, December.
    11. Barrutia, Jose M. & Echebarria, Carmen & Aguado-Moralejo, Itziar & Apaolaza-Ibáñez, Vanessa & Hartmann, Patrick, 2022. "Leading smart city projects: Government dynamic capabilities and public value creation," Technological Forecasting and Social Change, Elsevier, vol. 179(C).
    12. Olga Pilipczuk, 2020. "Sustainable Smart Cities and Energy Management: The Labor Market Perspective," Energies, MDPI, vol. 13(22), pages 1-24, November.
    13. Justyna Żywiołek & Francesco Schiavone, 2021. "Perception of the Quality of Smart City Solutions as a Sense of Residents’ Safety," Energies, MDPI, vol. 14(17), pages 1-16, September.
    14. Negar Noori & Thomas Hoppe & Martin de Jong, 2020. "Classifying Pathways for Smart City Development: Comparing Design, Governance and Implementation in Amsterdam, Barcelona, Dubai, and Abu Dhabi," Sustainability, MDPI, vol. 12(10), pages 1-24, May.
    15. Dagmara Kociuba & Mariusz Sagan & Waldemar Kociuba, 2023. "Toward the Smart City Ecosystem Model," Energies, MDPI, vol. 16(6), pages 1-26, March.
    16. Parul Gupta & Sumedha Chauhan & M. P. Jaiswal, 2019. "Classification of Smart City Research - a Descriptive Literature Review and Future Research Agenda," Information Systems Frontiers, Springer, vol. 21(3), pages 661-685, June.
    17. O’Dwyer, Edward & Pan, Indranil & Acha, Salvador & Shah, Nilay, 2019. "Smart energy systems for sustainable smart cities: Current developments, trends and future directions," Applied Energy, Elsevier, vol. 237(C), pages 581-597.
    18. Kyunam Kim & Jung-Kyu Jung & Jae Young Choi, 2016. "Impact of the Smart City Industry on the Korean National Economy: Input-Output Analysis," Sustainability, MDPI, vol. 8(7), pages 1-19, July.
    19. Liu, He & Li, Xuerong & Wang, Shouyang, 2021. "A bibliometric analysis of 30 years of platform research: Developing the research agenda for platforms, the associated technologies and social impacts," Technological Forecasting and Social Change, Elsevier, vol. 169(C).
    20. Haraguchi, Masahiko & Funahashi, Tomomi & Biljecki, Filip, 2024. "Assessing governance implications of city digital twin technology: A maturity model approach," Technological Forecasting and Social Change, Elsevier, vol. 204(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:igg:jswis0:v:19:y:2023:i:1:p:1-20. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Journal Editor (email available below). General contact details of provider: https://www.igi-global.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.