Visual Dialog is a multi-modal task that requires a model to participate in a multi-turn human dialog grounded on an image, and generate correct, human-like responses. In this paper, we propose a novel Adversarial Multimodal Feature Encoding (AMFE) framework for effective and robust auxiliary training of visual dialog systems. AMFE can force the language-encoding part of a model to generate hidden states in a distribution closely related to the distribution of real-world images, resulting in language features containing general knowledge from both modalities by nature, which can help generate both more correct and more general responses with reasonably low time cost. Experimental results show that AMFE can steadily bring performance gains to different models on different scales of data. Our method outperforms both the supervised learning baselines and other fine-tuning methods, achieving state-of-the-art results on most metrics of VisDial v0.5/v0.9 generative tasks. In recent years, there has been a rising attention in Artificial Intelligence on how to train a model to understand visual inputs from the physical world, and communicate them with human language. Typical problems include Visual Question Answering (VQA) (Antol et al., 2015) and Image Captioning (Xu et al., 2015). These tasks require a model to read an image and generate a proper response, such as answering a question grounded on the image, or generating a sentence to describe the image. As a more difficult extension, Visual Dialog (De Vries et al., 2017; Das et al., 2017a; Mostafazadeh et al., 2017) is a cluster of tasks featuring two agents conducting a multi-turn dialog grounded on an image. A model is usually trained  Corresponding Author to predict every single response of one agent in the two, based on the image and dialog history. There are also some different task settings such as directly training two agents to complete a goal-driven cooperative task such as Guessing Game (Das et al., 2017b). Tasks involving both the physical world (vi-sual images) and abstract world (languages) share a core issue: how to establish connections between these two worlds, and is there a framework to leverage these connections for learning? Temporarily, the majority of answers are learning end-to-end models with multi-modal feature fusion (Kim et al., 2016; Fukui et al., 2016; Yu et al., 2018). These methods usually merge the visual and language features into rich representations containing information from both sides. Some cross-modal attention methods (Lu et al., 2016; Nam et al., 2017) formulate the visual-language connections explicitly by parameterizing the attention weights to learn whether there is high correlation within certain pairs of language and visual feature vectors. However, in all these works, the merged representations or attention weights are only learned from pairwise (one image, one sentence) co-occurrence, and serve for the optimization of a loss function only related to the final ground-truth response. In fact, the features from both sides are not truly connected in an aspect of general distributions, but only merged into a new vector for each training/testing sample. We suppose that this is not good enough for a model to distill knowledge from both of the two worlds because the language/visual vectors do not contain knowledge from the other modality in the bottom level before they are merged. In this paper, we discuss another possibility. We want to establish an unsupervised framework of multi-modal encoding, which directly generates an image feature distribution from a language 2589 distribution, or vice versa. For example, when a neural network based model receives a natural language sentence x as input, it encodes x into a sequence of high-dimensional continuous vectors. All these language vectors can be projected into another latent space to have a new distribution p l . We train the language encoder to let the new distribution p l be the same as, or very close to, the distribution p v of all image features observed and encoded in the task data. Since we can partly recover a real-world image distribution from the language vectors achieved in this way, these language vectors intrinsically contain both language semantics and real-world image properties. This is a higher-level connection between the two worlds. In order to train a model to generate samples subject to a certain distribution p v from an original distribution p l , Generative Adversarial Networks (GANs) have been proved very effective (Goodfellow et al., 2014; Arjovsky et al., 2017; Miyato et al., 2018). Lample et al. (2018) used adversarial training on the vectors produced by sentence encoders for different languages in unsupervised machine translation. However, different languages in their task are in single modality and share encoder structures, making the same method not directly usable and extendable for multi-modal tasks with largely different prior distributions and complex encoder structures with attention. In our work, we propose Adversarial Multi-modal Feature Encoding (AMFE), a novel GAN-based training schedule with an attention-based sample-selecting method, which can successfully force the multi-modal vectors to have closely related distributions, benefitting the performances of various visual dialog systems. We test our method on the VisDial (Das et al., 2017a) benchmark (one example is shown in Table 1). A normal sample of VisDial contains an image and 10 turns of question-answering dialog from two people grounded on the image. A series of models have been proposed to solve the task, including memory and attention based models (Das et al., 2017a), reinforcement learning (Das et al., 2017b), knowledge transfer techniques (Lu et al., 2017) and GAN (Wu et al., 2017). Wu et al. (2017) designed a complex attention model and applied GAN in a traditional way to force the generated tokens to mimic real-world language (language vs. language), making their model only trainable through sequence Caption : Adogwithgogglesisina motorcyclesidecarA(1):canyoutellwhatkindofdogthisisB(1):helookslikebeautifulpitbullmix A(2): canyoutellifmotorcycleismoving orstill B(2): it'sparked A(3): isdog'stonguelollingout B(3): notreally Table 1: An example from VisDial dataset. sampling and reinforcement learning. Our work, on the other hand, applies a directly differentiable GAN on continuous vectors as a multi-modal feature encoding method (language vs. image). Our contributions include:  We propose AMFE: a novel Adversarial Multi-modal Feature Encoding framework to benefit visual dialog models. The core idea is to force features from different modalities to have closely related distributions. We develop efficient AMFE implementations, including a novel attention-based sample selecting method, for various commonly-used visual dialog models. Experimental results show that AMFE brings robust performance gains to different visual dialog models. We achieve state-of-the-arts on most metrics of VisDial v0.5/v0.9 generative tasks. Visual Dialog is a cluster of tasks sharing two properties: multi-turn and cross-modality. VisDial (Das et al., 2017a) is a widely-used benchmark with question-answering style dialogs grounded on real-world images. As a special case of dialog generation tasks, VisDial share some of the research concerns with single-modal natural language dialog generation (Dhingra et al., 2016; Serban et al., 2017; Sordoni et al., 2015; Serban et al., 2016; Liu et al., 2016). Natural language dialogs are usually discrete, state-dependent and style-free, thus some reinforcement learning (RL) methods have been proposed (Li et al., 2016). Das et al. (2017b) built an cooperative image guessing task on VisDial: they train both the questioner and the answerer, making them complete a same goal to help the questioner produce a guessing or imagination of the unseen image described by 2590 the answerer. The distance between the guessing and the target image is used as reward for reinforcement learning. In some extreme settings, such a task definition can even lead to emergence of a new language between robots (Kottur et al., 2017). After per-training, using their reinforcement learning method as an auxiliary loss can also bring performance gain in standard VisDial metrics such as mean rank. However, generating a reward based on just one target image for a training sample may lead to a kind of overfitting. Language is highly abstract: one dialog can correctly describe a lot of different scenes in real world, so why should we force a dialog to fit one single example among them? Therefore, generating a reward from adversarial training is a more efficient way because it goes beyond individual samples into distributions. There are two previous works (Wu et al., 2017; Lu et al., 2017) that use GAN-like methods to boost the performances of pre-trained VisDial models. (Wu et al., 2017) proposes to use adversarial reinforcement learning. A discriminator is trained to distinguish the tokens of real/generated answers, and the answerer (generator) is trained via RL using a reward related to the score given by discriminator. This method is very effective, but using both RL Monte Carlo and GAN brings high computational cost. Also, a lot of tricks are involved for a good training. Our method, on the other hand, does not need Monte-Carlo sampling to compute immediate reward while generating each of the N words in a sentence ( O ( N ) time cost). (Lu et al., 2017) uses a knowledge-transferring method between generative and discriminative task settings. However, this requires the models on both settings to be pre-trained well enough. Our work is also an adversarial learning based method, but it is more robust, time-efficient and effective. GAN (Goodfellow et al., 2014; Arjovsky et al., 2017; Miyato et al., 2018) has raised much attention because of its ability to directly generate samples subject to a target distribution. Many training techniques have been proposed to solve the unstable training problems of GAN (Gulrajani et al., 2017; Kurach et al., 2018). Wasserstein GAN (WGAN) (Arjovsky et al., 2017) is a successful method using critic learning loss and weight clipping operations. We borrow some ideas from WGAN in the adversarial training of our model. GAN well suits the image generation tasks because image signals are continuous and thus differentiable, enabling the gradient directly flowing back from the discriminator to generator. In language generation tasks, however, how to deal with the discrete sequence of symbols generated by the generator has long been a problem. A widely-used solution is applying RL with rewards generated by the discriminator (Wang et al., 2018; Li et al., 2017). As mentioned above, this is time-costing because RL needs to explore a large action space by sampling multiple action sequence. Besides, how the immediate reward is computed after generating each word is also a difficult problem. Another solution is to avoid the discrete problem by applying adversarial training on the hidden states of the generator. This requires that there is a known distribution p for the hidden states we want the model to generate. A successful case is reported by (Lample et al., 2018): using adversarial training to restrict the hidden states of source language and target language (both from vanilla LSTMs) into a same latent space can boost the performance of unsupervised machine translation. Our AMFE framework is also an adversarial training on the language hidden states, but we are the first to use this kind of methods to establish connections between different modalities. Our training procedure is also largely different from (Lample et al., 2018) with our modified WGAN-like algorithm and a novel attention-based sample selection method: they are critical for training convergency on multi-modal tasks, with complex attention-based model structures. We first define the task and our framework formally, and then describe how it is implemented and trained on different visual dialog models. In the VisDial task, each sample contains an image I , a caption sentence C and a dialog D with T = 10 turns in total. In each turn t , there is a question q t about the image, and a ground truth answer a t . The model needs to read the dialog history H = { C, ( q 1 , a 1 ) , ..., ( q t  1 , a t  1 ) } and image I , to generate an answer as a response to q t . We rewrite H t = ( q t , a t ) and H 0 = C . Formally, the dialog agent (named A-Bot) outputs an answer Image Encoding Language Encoding h v h l A: Is there a man or a woman? The goal of our Adversarial Multi-modal Feature Encoding (AMFE) is to restrict the distribution of feature representations from one modality m 1 to be closely related to that from another modality m 2 . We take m 1 = l ( anguange ) and m 2 = v ( isual ) . Specifically, A-Bot encodes language inputs into vectors h l , and visual inputs into h v , respectively. We want h l and h v to have indistinguishable distributions: h l , h v  p ( h ) . To achieve this goal, we use a discriminative model (named D-Bot) to classify whether a vector encoded by A-Bot comes from modality l or v . D-Bot is trained with real h l and h v samples, while A-Bot is trained to generate language vectors h l that can confuse D-Bot to classify them as label v . Figure 1 shows our framework. We implement our AMFE method on two commonly-used visual dialog models, using them as A-Bot. The two A-Bot models are named Hierarchical Recurrent Encoder (HRE) and History-Conditioned Image Attentive Encoder (HCIAE), respectively. A-Bot learns to predict the right answer in each turn. In this process, it also encodes language and visual inputs into h l and h v samples, which we use for AMFE training. HRE is a hierarchical LSTM (Hochreiter and Schmidhuber, 1997) model used in (Das et al., 2017a,b). In HRE, a pre-trained Convolutional Neural Network (CNN) encodes the image into a single feature vector, which is further mapped into a visual representation I by a trainable MultiLayer Perceptron (MLP). In each turn t , the question is encoded by a word-level LSTM into a question vector q t , and the dialog history in the previous step H t  1 is encoded by another LSTM into vector f t  1 . There is a state-tracker LST M st on the top level: LST M st is forwarded one step each turn, integrating all the encoded vectors mentioned above. It reads the encoded history f t  1 , image vector I , the question q t and the previous hidden state s t  1 from itself, and produces the new hidden state representation s t : s t = LST M st ([ q t , I, f t  1 ] , s t  1 ) , (3) where [  ] stands for concatenation. The answer decoder in HRE is an LSTM that takes s t as initial state, and predicts one word at a time by a softmax probability over the vocabulary, to generate the whole answer sentence. Figure 2(a) shows the encoder structure of HRE. We use image vectors I as h v samples (dark green) in AMFE, and both the q and f vectors as h l samples (pink). HCIAE model (Lu et al., 2017) contains an textual attention on all history vectors based on the question, and a visual attention based on both the history and the question. In detail, it uses a pre-trained CNN to encode the image into a set of visual feature vectors V . Each vector in V is further passed through a trainable MLP, resulting in a visual feature set { i 0 , ...i K  1 } . In each turn t , the question is encoded by an LSTM into vector q t ; the dialog history { H 0 , H 1 , ..., H t  1 } is encoded by another word-level LSTM into vectors { f 0 , ..., f t  1 } . The attention weight between q t and each history vector f j is computed as: z jt =  Ta tanh ( W f f j + W q q t ) ,  jt = softmax ( z jt ) , (4) where  Ta  R d  1 , W f  R d  d , W q  R d  d are trainable parameters; d is the length of both 2592 LSTM LSTM CNNI Q t H t-1 s t s t-1 s t+1 LSTM st Image Encoder Sentence Encoders [  ] h v candidates h l candidates question and history features. A memory vector  m t is computed by:  m t = t  1 (cid:2) j =0  jt f j . The memory vector is further used as a key to compute a similar visual attention over { i 0 , ...i k  1 } to achieve a final image vector  v t . The final output of the encoder is computed by: e t = tanh ( W e [ q t ,  m t ,  v t ]) , (6) where W e  R d  3 d is trainable parameters; [  ] stands for concatenation. The answer decoder is an LSTM like that of HRE, taking e t as input. Figure 2(b) shows the structure of HCIAE encoder. HCIAE produces more visual vectors for each image than HRE. We take all the q and f vectors as h l candidates of AMFE, and the spatial visual features { i 0 , ...i k  1 } as candidates for h v . Despite the multiple choices of A-Bot, our D-Bot is always an MLP with two hidden layers of size 512 and ReLU activation. It is used to compute a loss function that forces all the h l samples to be subject to the same distribution p ( h ) as the visual vectors h v . D-Bot takes a vector h in size d as its input, and predicts the probability of h coming from real image distribution and the visual encoder:  p v ( modality = v | h ) = DBot ( h ) . D-Bot is the discriminator from a GAN viewpoint. A-Bot must learn to confuse D-Bot in order to generate language features indistinguishable from image features. To train our model, we use standard supervised training with cross-entropy loss function for pretraining, and add in adversarial training to produce an auxiliary loss to improve feature encoding. The supervised learning loss is: L su = 1 NN (cid:2) n =1  log( p ( w tn | w t<n )) , (8) where N is the full length of the decoded sentence. (9) We sum L adv as an auxiliary loss with a tunable weight  , making A-Bot minimize: LG = L su + L adv . (10) On the other hand, D-Bot maximizes the following objective to distinguish real-world image vectors h v from the language vectors h l : LD = E h v [ DBot ( h v )]  E h l [ DBot ( h l )] . (11) We switch between A-Bot and D-Bot updates for each batch of dialog samples. For adversarial learning, A-Bot is trained to minimize the probability that D-Bot predicts the generated features to be fake samples. Following WGAN (Arjovsky et al., 2017), we do not use logarithm but directly optimize the likelihood itself: L adv =  E h l [ DBot ( h l )] . We have specified where the h l and h v samples come from while using different A-Bots in Section 3.3. Typically, for each batch of samples with batch-size M , in each turn t , there are M question vectors and M  t history vectors as h l candidates. For HRE encoder, there are M image vectors as h v candidates, while the number is M  K for the HCIAE encoder; K is the number of pixels in the final CNN feature-map. Thus, it is impossible to use all the generated samples in AMFE. For a successful training, the selected samples must be efficient, informative and balanced. While using HCIAE, in order to compute L adv , we use M question vectors and M  w history vectors as h l samples. The history vectors are selected using textual attention weights  jt produced by the temporary model: for each dialog, we pick the top w history vectors with the highest attention weights. We call this Attention-based Sample Selection (AbS). While computing LD to train DBot, we use the same technique on the image, using the top-attended M  w image vectors, together with another M image vectors randomly sampled from the dataset as positive samples h v . The M question vectors and M  w history vectors are used as a pool of negative samples h l . In our experiments, w = 1 , 2 works well. While using HRE, since the model always at-tends on f t  1 by default (Eq. 3), we directly select q t and f t  1 as h l samples. We use the M image vectors I in this batch, together with another M image vectors randomly sampled from the dataset as the pool of h v . The full training procedure is specified in Algorithm 1. 4 Experiments 4.1 The VisDial Dataset VisDial is a visual dialog dataset based on MS COCO (Lin et al., 2014) images. There are 10 turns of human-posed question-answering dialogs on each image, with the questioner kept not seeing the image during the data collection process. For generative models, a model must give the probability of generating each candidate answer without seeing other candidates, and the rank of the ground-truth answer in the 100 candidates is used to compute different evaluation metrics; for discriminative models, the model can read and encode all the candidate answers and directly assign scores on them. According to the nature of GANs Algorithm 1 AMFE Training Procedure. and similarities to real-world application scenarios, we use the generative setting for our model: it is equipped with a sequential decoder instead of a scoring module. For fair and sufficient comparison, we evaluate our model on both VisDial v0.5 and VisDial v0.9. VisDial v0.5 has 68k COCO images, for a total of 680k QA-pairs. Following (Das et al., 2017a) and (Das et al., 2017b), we use 50,729 images for training, 7,663 for validation and 9,628 for testing. Visdial v0.9 has 123,287 images. There are different splitting of train/valid/test in previous work. We follow (Lu et al., 2017) to use 82k for training, 1k for validation and 40k for testing. 1 We compare our results to several existing models on the VisDial dataset, including:  Answer Prior (Das et al., 2017a): directly encoding answer candidates with an LSTM and scoring by a linear model that captures the frequency of answers in the training set. NN-QI (Das et al., 2017a): a k-Nearest Neighborhood method considering only the question and the image. Unlike generative methods, both Answer Prior and NN-QI need to know the answer candidates. LF-QIH-G (Das et al., 2017a): a Late Fusion encoder that encodes the question, image and history separately. The encoded features are concatenated and linearly transformed to a 1 VisDial has released v1.0 recently, and claims that models trained on v0.9 should also use the new v1.0 test set. Due to lack of baselines in the generative task, we follow the original widely-used settings of v0.5 and v0.9. joint representation. The answer is produced by a generative decoder. HRE (Das et al., 2017b): the HRE model introduced in Section 3.3. HREA-QIH-G (Das et al., 2017a): a modified HRE A-Bot with attention to dialog history. MN-QIH-G (Das et al., 2017a): a Memory Network encoder that stores each piece of dialog history embeddings in an explicit memory. These embeddings can be attended and fused while generating the answer. HCIAE (Lu et al., 2017): the HCIAE model introduced in Section 3.3. CoAtt (Wu et al., 2017): this is a previous state-of-the-art model with a more complex co-attention encoder; the decoder is enhanced by adversarial reinforcement learning for better answer generation. We first test the efficiency of AMFE on the simpler A-Bot model: HRE. We use VisDial v0.5 as our benchmark for fair comparison with other HRE-based models and auxiliary training methods. For Visdial v0.5 dataset, we follow the preprocessing procedure and hyper-parameters described in (Das et al., 2017b). We pass each image through a pre-trained VGG-16 (Simonyan and Zisserman, 2015) CNN, and pick the single f7 vector as input image feature. We limit the maximum lengths of captions, questions and answers to be 40, 20 and 20, respectively; we remove words appearing less than 5 times in the training set, and replace them by a UNK token. We use vector size 300 for word embedding and 512 for all language and visual feature vectors. All LSTMs have two layers. We pre-train A-Bot with L su for 20 epochs before L adv is added in. The batch-size is set to be 32. After each update of A-Bot, we perform 5 DBot updates. We use the 32 encoded image vectors in the batch, together with 32 image vectors randomly sampled from the dataset, to form 64 positive samples; for negative samples, we use the 32 question vectors and 32 history vectors ( t  1 ) from the updated A-Bot. We use Adam (Kingma and Ba, 2014) for A-Bot and RMSprop (Tieleman and Hinton, 2012) algorithm for D-Bot to perform gradient descending. The learning rate is set to 1e-3 Model MRR R@1 R@5 R@10 Mean Answer Prior 0.311 19.85 39.14 44.28 31.56 NN-QI 0.385 29.71 46.57 49.86 30.90 LF-QIH-G 0.430 33.27 51.96 58.09 23.04 HREA-QIH-G 0.442 34.47 53.43 59.73 21.83 MN-QIH-G 0.443 34.62 53.74 60.18 21.69 HRE-MLE 0.436 33.02 53.41 60.09 21.83 Frozen-Q-Multi 0.437 33.22 53.67 60.48 21.13 HRE-AMFE 0.445 34.62 53.95 60.76 20.98 Table 2: VisDial v0.5 evaluation results. for pre-training, further decayed to 5e-5; after adversarial training starts, the learning rate is fixed to 5e-5 for both Aand D-Bot. In the weight clipping step of WGAN (Arjovsky et al., 2017), we use a clipping parameter c = 0 . 01 . On VisDial v0.5, two previous top models are a Memory Network based model (MN-QIH-G) by (Das et al., 2017a) and a multi-loss training on HRE encoder (Frozen-Q-Multi) based on goal-driven reinforcement learning (Das et al., 2017b). We start from the same HRE hyper-parameters and checkpoint as (Das et al., 2017b), but continue with our AMFE instead of reinforcement learning. Table 2 shows the results on all the five evaluation metrics on VisDial v0.5. Results in the first 4 rows are copied from (Das et al., 2017a). AMFE achieves better performances than the supervised training of A-Bot model (HRE-MLE), especially significant on R@5, R@10 and mean rank, indicating that the adversarial feature encoding results in generally better dialogs. It also outperforms the another HRE-like model with history attentions (HREA-QIH-G). While used for multi-loss training, AMFE is significantly better than Frozen-Q-Multi, setting a new state-of-the-art on all metrics. We point out that in Frozen-Q-Multi (Das et al., 2017b), the goal-driven reinforcement leaning reward is computed pair-wise (consider-ing how much can the questioner rebuild the image from the answerer's words), but the reward computed with a single image is not good enough to evaluate the dialog actions. This is because language is much more abstract than image, and failure to recover an image does not necessarily mean that the dialog is actually bad. Our method could avoid this issue because adversarial training is based on general distributions. In this section, we test the efficiency of AMFE for the HCIAE model with attention. We use VisDial v0.9 as our benchmark for fair comparison with (Lu et al., 2017). For Visdial v0.9 dataset, we follow the preprocessing procedure and HCIAE structure described in (Lu et al., 2017). We pass each image through a pre-trained VGG-19 CNN, resulting in a 512  7  7 feature-map as visual input. To speed up convergence, we add a Batch Normalization (Ioffe and Szegedy, 2015) after the MLP that further encodes these visual vectors. We limit the maximum lengths of captions, questions and answers to be 24, 16 and 8, respectively. All LSTMs have only one layer. HCIAE can be trained with either supervised loss (HCIAE-G-MLE) or with multi-loss involving knowledge-transfer (HCIAE-G-DIS). We test AMFE in both settings. For HCIAE-G-MLE, we pre-train HCIAE model with supervised loss for 20 epochs using learning rate 4e-4, and switch to AMFE training with learning rate 5e-5. For HCIAE-G-DIS, we start from the generative model trained with AMFE, together with a pre-trained HCIAE discriminative model. We follow the original knowledge-transfer training schedule, and add our L adv to the original mixed loss function with weight 1. We use batch-size 32 for AMFE training, although the original paper used 128. Other settings are kept the same. For more details please see (Lu et al., 2017). Table 3 shows the results on v0.9. All the HCIAE results are picked from (Lu et al., 2017), and all CoAtt results are picked from (Wu et al., 2017); CoAtt-GAN-TF stands for training a CoAtt model with adversarial reinforcement learning and supervised teacher-forcing; HCIAE-AMFE stands for using AMFE on an HCIAE-G-MLE pre-trained model; HCIAE-GD-AMFE means using AMFE as an additional loss to join the HCIAE-G-DIS multi-loss training. On VisDial v0.9, we observe that using AMFE on HCIAE can also boost the performances. Comparing HCIAE-G-MLE and HCIAE-AMFE, we can observe the same advantage over supervised training as on HRE, indicating that our method works for different dataset scales and A-Bot struc-Model MRR R@1 R@5 R@10 Mean Answer Prior 0.374 23.55 48.52 53.23 26.50 NN-QI 0.427 33.13 50.83 58.69 19.62 LF-QIH-G 0.520 41.83 61.78 67.59 17.07 HREA-QIH-G 0.524 42.28 62.33 68.17 16.79 MN-QIH-G 0.526 42.29 62.85 68.88 17.06 CoAtt-G-MLE 0.541 44.32 63.82 69.75 16.47 CoAtt-GAN-TF 0.558 46.10 65.69 71.74 14.43 HCIAE-G-MLE 0.539 44.06 63.55 69.24 16.01 HCIAE-G-DIS 0.546 44.35 65.28 71.55 14.23 HCIAE-AMFE 0.547 44.40 65.35 71.69 14.42 HCIAE-GD-AMFE 0.554 45.42 66.09 72.30 14.11 Table 3: VisDial v0.9 evaluation results. tures; comparing HCIAE-AMFE and HCIAE-G-DIS, AMFE is a competitive method for auxiliary training. Combining AMFE and HCIAE-G-DIS achieves better results than previous state-of-the-art (Wu et al., 2017) on R@5, R@10 and mean rank, and comparable on MRR and R@1. Besides, AMFE trains reasonably faster because we avoid the O ( N ) time cost for Monte-Carlo sampling while computing temporary rewards (Wu et al., 2017). We explain the efficiency of AMFE in two aspects. Firstly, AMFE is an adversarial training procedure forcing the language to be encoded into a distribution closely connected to the images. With attention-based sample selection, the most informative samples from both modalities are able to transfer knowledge. Secondly, like Batch Normalization, AMFE contributes to bring better numerical properties to the intermediate tensors in a network, especially on their means and variances, which could potentially benefit model performance. Both the weight of adversarial loss and the attention-based sample selection are critical to good performance. Table 4 shows ablation studies on these factors on HCIAE and VisDial v0.9. The above results show that AMFE is especially strong at more general metrics such as R@5 and mean rank. To confirm that adversarial training on hidden states can help much to generate responses that are more natural, we randomly select 100 dialog samples from both VisDial v0.5 and v0.9 dataset, and ask two human subjects to vote for the responses generated by two groups of models: HRE-MLE vs. HRE-AMFE on v0.5, and HCIAE-G-MLE vs. HCIAE-AMFE on v0.9, both with beam-size 5. Model names are hidden while voting. We ask the human subjects to consider two metrics separately: (1) the fluency of the generated answer sentences and (2) the correctness of the answers compared to the ground-truths. As shown in Table 6, AMFE wins all the votes with different metric and different models, indicating that AMFE is robust in generating more natural responses. We randomly sample some dialogs from VisDial v0.5 validation set and illustrate the ground-truth answers and the generated answers with/without AMFE. Two results are shown in Table 5. In the first example, the model trained with AMFE generates a right vs. wrong answer in the 8-th turn, and a grammatically better response in the 5-th turn, compared to supervised pre-training. In the second example, the model trained with AMFE has a generally right understanding of the questions and the image, while the HRE-MLE model is generating response as if it does not see the image. This indicates that encoding language features in the image space leads to better understanding on both modalities. We propose AMFE: an unsupervised multi-modal feature encoding framework and its implementations on different commonly-used visual dialog models. Our core idea is to force features from different modalities to have closely related distributions. Experiments show that AMFE can bring performance gains to both simple and complex models on different scales of VisDial dataset. Future work will possibly be visualizing the visual and language features encoded by AMFE to find more straightforward interpretations, as well as trying our method on more complex structures, discriminative models, and on discriminative tasks such as VQA and visual reasoning. We thank the reviewers for their insightful comments. This work was supported by the National Natural Science Foundation of China (61602479), the Beijing Brain Science Project (Z181100001518006) and the Strategic Priority Research Program of the Chinese Academy of Sciences (XDB32070000).
abstain objective abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain objective result abstain abstain abstain abstain objective result abstain abstain abstain abstain method objective objective objective objective result other other other other other other other other other method other other other other other other method other other abstain other other other abstain other other other other other other method other objective abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain method method abstain method abstain abstain method method abstain abstain abstain abstain abstain method method abstain method abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain method method method method method abstain method abstain method method method other abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain method method abstain abstain abstain method method method method abstain abstain abstain other result result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain method abstain abstain abstain abstain objective method abstain result other other