Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model
We present a novel training framework for neural sequence models, particularly for grounded dialog generation. The standard training paradigm for these models is maximum likelihood estimation (MLE), or minimizing the cross-entropy of the human responses. Across a variety of domains, a recurring problem with MLE trained generative neural dialog models (G) is that they tend to produce 'safe' and generic responses ('I don't know', 'I can't tell'). In contrast, discriminative dialog models (D) that are trained to rank a list of candidate human responses outperform their generative counterparts; in terms of automatic metrics, diversity, and informativeness of the responses. However, D is not useful in practice since it can not be deployed to have real conversations with users. Our work aims to achieve the best of both worlds -the practical usefulness of G and the strong performance of D -via knowledge transfer from D to G. Our primary contribution is an end-to-end trainable generative visual dialog model, where G receives gradients from D as a perceptual (not adversarial) loss of the sequence sampled from G. We leverage the recently proposed Gumbel-Softmax (GS) approximation to the discrete distribution -specifically, a RNN augmented with a sequence of GS samplers, coupled with the straight-through gradient estimator to enable end-to-end differentiability. We also introduce a stronger encoder for visual dialog, and employ a self-attention mechanism for answer encoding along with a metric learning loss to aid D in better capturing semantic similarities in answer responses. Overall, our proposed model outperforms state-of-the-art on the VisDial dataset by a significant margin (2.67% on recall@10).
This paper describes an improved training procedure for visual dialogue models.
Rather than maximizing the likelihood of a collection of training captions, this
approach first trains a discriminator model to rank captions in a given context
by embedding them in a common space, then uses scores from this discriminator as
an extra component in a loss function for a generative sequence prediction
model. This improved trainin procedure produces modest improvements on an
established visual dialogue benchmark over both previous generative approaches
as well as adversarial training.
I think this is a pretty good paper, though there are a few places in which
the presentation could be improved.
SPECIFIC COMMENTS
The introduction claims that the discriminator "has access to more information
than the generator". I'm not clear on what this means. It is
true that in a single update step, the discriminator here is trained to
distinguish the ground truth answer from a collection of answers, while in the
standard conditional GAN setting the discriminator compares only one sample at a
time to the ground truth. But over the course of training a normal GAN
discriminator will also get access to lots of samples from the training
distribution. More generally, there's an equivalence between the problem of
learning to assign the right probability to a sample from a distribution and
learning to classify whether a sample came from the target distribution or a
noise distribution (see e.g. the NCE literature). Based on this section, I
expected the discriminator to be able to learn features on interactions between
multiple candidates, and was confused when it wound up assigning an embedding to
each candidate independently.
Related work: given the offered interpretation of D as a "perceptual loss", I
was surprised not to see any mention in the related work section of similar loss
functions used image generation tasks (e.g. Johnson et al). I was not able to
find anything in a quick lit search of similar losses being used for natural
language processing tasks, which is one of the things that's exciting about this
paper, but you should still discuss similar work in other areas.
@137 it's standard form
The explanation offered for the objective function at 177 is a little strange.
This is a logistic regression loss which encourages the model to assign highest
probability to the ground-truth caption, and I think it will be most
understandable to readers if presented in that familiar form. In particular, it
does not have an interpretation as a metric-learning procedure unless the norms
of the learned representations are fixed (which doesn't seem to be happening
here), and is not a margin loss.
I don't understand what "assigned on-the-fly" @186 means. Just that
you're reusing negative examples across multiple examples in the same batch?
@284 modifications to encoder alone
Table captions should explain that MRR -> "mean reciprocal rank" and especially
"mean" -> "mean rank".