Author
Listed:
- Longbin Jin
(Artificial Intelligence & Computer Vision Lab., Konkuk University, Seoul 05029, Republic of Korea)
- Hyuntaek Jung
(Artificial Intelligence & Computer Vision Lab., Konkuk University, Seoul 05029, Republic of Korea)
- Hyo Jin Jon
(Artificial Intelligence & Computer Vision Lab., Konkuk University, Seoul 05029, Republic of Korea)
- Eun Yi Kim
(Artificial Intelligence & Computer Vision Lab., Konkuk University, Seoul 05029, Republic of Korea)
Abstract
Large-scale Visual-Language Models have demonstrated powerful adaptability in video recognition tasks. However, existing methods typically rely on fine-tuning or text prompt tuning. In this paper, we propose a visual-only prompting method that employs watermark and trademark prompts to bridge the distribution gap of spatial-temporal video data with Visual-Language Models. Our watermark prompts, designed by a trainable prompt generator, are customized for each video clip. Unlike conventional visual prompts that often exhibit noise signals, watermark prompts are intentionally designed to be imperceptible, ensuring they are not misinterpreted as an adversarial attack. The trademark prompts, bespoke for each video domain, establish the identity of specific video types. Integrating watermark prompts into video frames and prepending trademark prompts to per-frame embeddings significantly boosts the capability of the Visual-Language Model to understand video. Notably, our approach improves the adaptability of the CLIP model to various video action recognition datasets, achieving performance gains of 16.8%, 18.4%, and 13.8% on HMDB-51, UCF-101, and the egocentric dataset EPIC-Kitchen-100, respectively. Additionally, our visual-only prompting method demonstrates competitive performance compared with existing fine-tuning and adaptation methods while requiring fewer learnable parameters. Moreover, through extensive ablation studies, we find the optimal balance between imperceptibility and adaptability. Code will be made available.
Suggested Citation
Longbin Jin & Hyuntaek Jung & Hyo Jin Jon & Eun Yi Kim, 2025.
"Watermark and Trademark Prompts Boost Video Action Recognition in Visual-Language Models,"
Mathematics, MDPI, vol. 13(9), pages 1-15, April.
Handle:
RePEc:gam:jmathe:v:13:y:2025:i:9:p:1365-:d:1639696
Download full text from publisher
Corrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:13:y:2025:i:9:p:1365-:d:1639696. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
We have no bibliographic references for this item. You can help adding them by using this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .
Please note that corrections may take a couple of weeks to filter through
the various RePEc services.