What is an example of a common sense question?
Can you park here?
Is something under the sink broken?
Does this man have children?
The Visual Question Answering (VQA) task combines challenges for processing data with both Visual and Linguistic processing, to answer basic `common sense' questions about given images. Given an image and a question in natural language, the VQA system tries to find the correct answer to it using visual elements of the image and inference gathered from textual questions. In this survey, we cover and discuss the recent datasets released in the VQA domain dealing with various types of question-formats and enabling robustness of the machine-learning models. Next, we discuss about new deep learning models that have shown promising results over the VQA datasets. At the end, we present and discuss some of the results computed by us over the vanilla VQA models, Stacked Attention Network and the VQA Challenge 2017 winner model. We also provide the detailed analysis along with the challenges and future research directions.
Visual Question Answering (VQA) refers to a challenging task which lies at the intersection of image understanding and language processing. The VQA task has witnessed a significant progress in the recent years by the machine intelligence community. The aim of VQA is to develop a system to answer specific questions about an input image. The answer could be in any of the following forms: a word, a phrase, binary answer, multiple choice answer, or a fill in the blank answer. Agarwal et al. BIBREF0 presented a novel way of combining computer vision and natural language processing concepts of to achieve Visual Grounded Dialogue, a system mimicking the human understanding of the environment with the use of visual observation and language understanding. The advancements in the field of deep learning have certainly helped to develop systems for the task of Image Question Answering. Krizhevsky et al BIBREF1 proposed the AlexNet model, which created a revolution in the computer vision domain. The paper introduced the concept of Convolution Neural Networks (CNN) to the mainstream computer vision application. Later many authors have worked on CNN, which has resulted in robust, deep learning models like VGGNet BIBREF2, Inception BIBREF3, ResNet BIBREF4, and etc. Similarly, the recent advancements in natural language processing area based on deep learning have improved the text understanding prforance as well. The first major algorithm in the context of text processing is considered to be the Recurrent Neural Networks (RNN) BIBREF5 which introduced the concept of prior context for time series based data. This architecture helped the growth of machine text understanding which gave new boundaries to machine translation, text classification and contextual understanding. Another major breakthrough in the domain was the introduction of Long-Short Term Memory (LSTM) architecture BIBREF6 which improvised over the RNN by introducing a context cell which stores the prior relevant information. The vanilla VQA model BIBREF0 used a combination of VGGNet BIBREF2 and LSTM BIBREF6. This model has been revised over the years, employing newer architectures and mathematical formulations. Along with this, many authors have worked on producing datasets for eliminating bias, strengthening the performance of the model by robust question-answer pairs which try to cover the various types of questions, testing the visual and language understanding of the system. In this survey, first we cover major datasets published for validating the Visual Question Answering task, such as VQA dataset BIBREF0, DAQUAR BIBREF7, Visual7W BIBREF8 and most recent datasets up to 2019 include Tally-QA BIBREF9 and KVQA BIBREF10. Next, we discuss the state-of-the-art architectures designed for the task of Visual Question Answering such as Vanilla VQA BIBREF0, Stacked Attention Networks BIBREF11 and Pythia v1.0 BIBREF12. Next we present some of our computed results over the three architectures: vanilla VQA model BIBREF0, Stacked Attention Network (SAN) BIBREF11 and Teney et al. model BIBREF13. Finally, we discuss the observations and future directions.