In this paper, we propose a modular approach which separates the instruction-toaction mapping procedure into two separate stages. The two stages are bridged via an intermediate representation called a goal. The first stage maps an input instruction to a goal, while the second stage maps the goal to an appropriate policy selected from a set of robot policies. The policy is selected with an aim to guide the robot to reach the goal as close as possible. We implement the above two stages as a framework consisting of two distinct modules: an instruction-goal mapping module and a goal-policy mapping module. Given a human instruction in the evaluation phase, the instruction-goal mapping module first translates the instruction to a robot-interpretable goal. Once a goal is derived by the instructiongoal mapping module, the goal-policy mapping module then follows up to search through the goal-policy pairs to look for policy to be mapped by the instruction. Our experimental results show that the proposed method is able to learn an effective instruction-to-action mapping procedure in an environment with a given instruction set more efficiently than the baselines. In addition to the impressive data-efficiency, the results also show that our method can be adapted to a new instruction set and a new robot action space much faster than the baselines. The evidence suggests that our modular approach does lead to better adaptability and efficiency.
This paper presents an instruction-following model consisting of two modules: a
goal-prediction model that maps commands to goal representations, and an
execution model that maps goal representations to policies. The second module is
trained without command supervision via a goal exploration process, while the
first module is trained supervisedly in a metric learning framework.
This paper contains an important core insight---much of what's hard about
instruction following is generic planning behavior that doesn't depend on the
semantics of instructions, and pre-learning this behavior makes it possible to
use natural language supervision more effectively. However, the paper also
contains a number of serious evaluation and presentation issues. It is obviously
not ready to publish (uncaptioned figures, paragraphs interrupted mid-sentence,
etc.) and should not have been submitted to ICLR in its present form.
SUPERVISION AND COMPARISONS
I found comparisons between supervision conditions in this paper difficult to
understand. It is claimed that the natural language instruction following
approaches described in the first paragraph "require a large amount of human
supervision" in the form of action sequences. This is not exactly true, as some
approaches (e.g. Artzi 2013), can be trained with only task completion signals.
More problematically, all these approaches are contrasted with reinforcement and
imitation learning approaches, which are claimed to use "little human
supervision". In fact, most of the approaches listed in this section use exactly
the same supervision---either action sequences (imitation learning) or task
completion signals (reinforcement learning). Indeed, the primary distinction is
that the "NLP-style" approaches are typically evaluated on their ability to
generalize to new instructions, while the "RL-style" approaches are evaluated on
the (easier) problem of fitting the complete instruction distribution as quickly
as possible.
This confusion carries into the evaluation of the approach proposed in this
paper, which is compared to RL and IL baselines. It's hard to tell from the
text, but it appears that this is an "RL-style" evaluation setting, where we
only care about rapid convergence rather than generalization. But the baselines
are inadequately described, and it's not clear to me that they condition on the
commands at all. More significantly, it's not clear what an evaluation based on
"timesteps" means for a behavior-cloning approach---is this the number of
distinct trajectories observed? The number of gradient steps taken? Without
these explanations it is impossible to interpret the experimental results.
GENERALITY OF PROPOSED APPROACH
Despite the advantages of the high-level two-phase model proposed, the specific
implementation in this paper has two significant shortcomings:
- No evidence that it works with real language: despite numerous claims
  throughout the paper that the model is designed to interpret "human
  instructions", it is revealed on p7 that these instructions consist of one or two
  5-way indicator features. This is an extremely impoverished instruction space,
  especially compared to the numerous papers cited in the introduction that make
  use of large datasets of complex natural-language strings generated by human
  annotators. The present experiments do not support the use of the word "human"
  anywhere in the paper.
- No support for combinatorial action spaces. Even if we set aside the
  distinctions between human-generated instructions and synthetic command
  languages like used in Hermann Hill & al., the goal -> policy module is
  defined by a buffer of cached trajectories and goal representations. While
  this works for the simple environments considered in this paper, it cannot
  generalize to real-world instruction-following scenarios where the number of
  distinct goal configurations is too large to tractably enumerate. Again, this
  is a shortcoming that existing approaches do not suffer from (given
  appropriate assumptions about the structure of goal space), so the lack of
  comparisons is problematic.
CLARITY
The whole paper would benefit from copy-editing by an experienced English
speaker, but a few sections are particularly problematic:
- The first paragraph of 4.1.1 is extremely difficult to understand What does
  the fingertip do? What exactly is the action space?
- The end of the second paragraph is also difficult to understand; after reading
  it I still don't know what the extra "position" targets do.
- 4.1.4 is cut off mid-way through a sentence.
- last sentence of 4.2
The figures are also impossible to interpret: three of the four are captioned
"overview of the proposed framework", and none are titled.