Towards Fusion of Textual and Visual Modalities for Describing Audiovisual Documents

Audiovisual documents provide a wide range of content description through more descriptors from different media types. Indeed, the extraction of these descriptions has received an increasing attention. But, the lack of semantic description always persists. In fact, this lack affects the retrieval process. To address this problem, this paper describes an automatic and semantic description of cinematic audiovisual documents. This description is based not only on the audiovisual flux in this post-production phase but also in the documentation in the pre-production phase by using textual and visual modalities. In this context, to extract content description, we find it is essential to extract texts superposed in the image. This process is mainly based on the neural network classifier. Moreover, an effective OCR (Tesseract) is adapted for texts recognition. Experiments results confirmed the interesting performance through two databases, namely, “ICDAR 2011” and our own created database from the Internet Movie Database Imdb.