BVideoQA: Online English/Chinese bilingual video question answering

This article presents a bilingual video question answering (QA) system, namely BVideoQA, which allows users to retrieve Chinese videos through English or Chinese natural language questions. Our method first extracts an optimal one‐to‐one string pattern matching according to the proposed dense and long N‐gram match. On the basis of the matched string patterns, it gives a passage score based on our term‐weighting scheme. The main contributions of this approach to multimedia information retrieval literatures include: (a) development of a truly bilingual video QA system, (b) presentation of a robust bilingual passage retrieval algorithm to handle no‐word‐boundary languages such as Chinese and Japanese, (c) development of a large‐scale bilingual video QA corpus for system evaluation, and (d) comparisons of seven top‐performing retrieval methods under the fair conditions. The experimental studies indicate that our method is superior to other existing approaches in terms of precision and main rank reciprocal rates. When ported to English, encouraging empirical results also are obtained. Our method is very important to Asian‐like languages since the development of a word tokenizer is optional.