In sequence-to-sequence learning, the attention mechanism has been a great success in bridging the information between the encoder and the decoder. However, it is often overlooked that the decoder only has a single view of the source sequences, that is , the representations generated by the last encoder layer , which is supposed to be a  global view of source sequences . Such implementation hinders the decoder from concrete, fine-grained , local source information . In this work, we explore to reuse the representations from different encoder layers for layer-wise cross-view decoding , that is, different views of the source sequences are presented to different decoder layers . We investigate multiple , representative strategies for cross-view coding, of which the granularity consistent attention (GCA) strategy proves the most efficient and effective in the experiments on  neural machine translation task . Especially, GCA surpasses the previous state-of-the-art architecture on three machine translation datasets.
In sequence-to-sequence learning, the attention mechanism has been a great success in bridging the information between the encoder and the decoder. However, it is often overlooked that the decoder obtains only a single view of the source sequences, i.e. , the representations generated by the last encoder layer . Although those representations are supposed to be a comprehensive, global view of source sequences , such practice keeps the decoders from concrete, fine-grained source information generated by other encoder layers . In this work, we propose to encourage the decoder to take the full advantage of the multi-level source representations for layer-wise cross-view decoding . Concretely, different views of the source sequences are presented to different decoder layers and multiple strategies are explored to route the source representations. In particular, the granularity consistent attention (GCA) strategy proves the most efficient and effective in the experiments on the neural machine translation task , surpassing the previous state-of-the-art architecture on three benchmark datasets.
R
only has
obtains only
R
that is
i.e.
R
, which is
. Although those representations are
A
comprehensive,
R
. Such implementation hinders the decoder
, such practice keeps the decoders
R
, local source information
source information generated by other encoder layers
R
explore to reuse the representations from different encoder layers
propose to encourage the decoder to take the full advantage of the multi-level source representations
R
, that is,
. Concretely,
R
. We investigate multiple , representative strategies for cross-view coding, of which
and multiple strategies are explored to route the source representations. In particular,
A
the
R
. Especially, GCA surpasses
, surpassing
R
machine translation
benchmark