Recent efforts on training visual navigation agents conditioned on language using deep reinforcement learning have been successful in learning policies for two different tasks: learning to follow navigational instructions and embodied question answering. In this paper, we aim to learn a multitask model capable of jointly learning both tasks, and transferring knowledge of words and their grounding in visual objects across tasks. The proposed model uses a novel Dual-Attention unit to disentangle the knowledge of words in the textual representations and visual objects in the visual representations, and align them with each other. This disentangled task-invariant alignment of representations facilitates grounding and knowledge transfer across both tasks. We show that the proposed model outperforms a range of baselines on both tasks in simulated 3D environments. We also show that this disentanglement of representations makes our model modular, interpretable, and allows for transfer to instructions containing new words by leveraging object detectors.
The authors propose a multitask model using a novel “dual-attention” unit for embodied question answering (EQA) and semantic goal navigation (SGN) in the virtual game environment ViZDoom. They outperform a number of baseline models originally developed for EQA and SGN (but trained and evaluated in this multitask paradigm).
Comments and questions on the model and evaluation follow.
1. Zero-shot transfer claim:
1a. This is not really zero-shot transfer, is it? You need to derive object detectors for the meanings of the novel words (“red” and “pillar” from the example in the paper). It seems like this behavior is supported directly in the structure of the model, which is great — but I don’t think it can be called “zero-shot” inference. Let me know if I’ve misunderstood!
1b. Why is this evaluated only for SGN and not for EQA?
2. Dual attention module:
2a. The gated attention model only makes sense for inputs in which objects or properties (the things picked out by convolutional filters) are cued by single words. Are there examples in the dataset where this constraint hold (e.g. negated properties like “not red”)? How does the model do? (How do you expect to scale this model to more naturalistic datasets with this strong constraint?)
2b. A critical claim of the paper is that the model learns to “align the words in both the tasks and transfer knowledge across tasks.” (Earlier in the paper, the claim is that “This forces the convolutional network to encode all the information required with respect to a certain word in the corresponding output channel.”) I was expecting you would show some gated-attention visualizations (not spatial-attention visualizations, which are downstream) to back up this claim. Can you show me visualizations of the gated-attention weights (especially when trained on the No-Aux task) which demonstrate that words and objects/properties in the images have been properly aligned? Show that e.g. the filter at index i only picks out objects/properties cued by word i?
3. Auxiliary objective: it seems like this objective solves most of the language understanding problem relevant in this task. Can you motivate why it is necessary? What is missing in the No-Aux condition, exactly? Is it just an issue with PPO optimization? Can you do error analysis on No-Aux to motivate the use of the Aux task?
4. Minor notes:
4a. In appendix A, the action is labeled “Turn Left” but the frames seem to suggest that the agent is turning right.
4b. How are the shaded regions estimated in figs. 7, 8? They are barely visible — are your models indeed that consistent across training runs? (This isn’t what I’d expect from an RL model! This is true even for No-Aux..?)
4c. Can you make it clear (via bolding or coloring, perhaps) which words are out-of-vocabulary in Table 3? (I assume “largest” and “smallest” aren’t OOV, for example?)