In sequence-to-sequence learning, the attention mechanism has been a great success in bridging the information between the encoder and the decoder. However, it is often overlooked that the decoder only has a single view of the source sequences, that is , the representations generated by the last encoder layer , which is supposed to be a  global view of source sequences . Such implementation hinders the decoder from concrete, fine-grained , local source information . In this work, we explore to reuse the representations from different encoder layers for layer-wise cross-view decoding , that is, different views of the source sequences are presented to different decoder layers . We investigate multiple , representative strategies for cross-view coding, of which the granularity consistent attention (GCA) strategy proves the most efficient and effective in the experiments on  neural machine translation task . Especially, GCA surpasses the previous state-of-the-art architecture on three machine translation datasets.
In sequence-to-sequence learning, the attention mechanism has been a great success in bridging the information between the encoder and the decoder. However, it is often overlooked that the decoder obtains only a single view of the source sequences, i.e. , the representations generated by the last encoder layer . Although those representations are supposed to be a comprehensive, global view of source sequences , such practice keeps the decoders from concrete, fine-grained source information generated by other encoder layers . In this work, we propose to encourage the decoder to take the full advantage of the multi-level source representations for layer-wise cross-view decoding . Concretely, different views of the source sequences are presented to different decoder layers and multiple strategies are explored to route the source representations. In particular, the granularity consistent attention (GCA) strategy proves the most efficient and effective in the experiments on the neural machine translation task , surpassing the previous state-of-the-art architecture on three benchmark datasets.
R
only has
obtains only
clarity
clarity clarity clarity
R
that is
i.e.
clarity
clarity style clarity
R
, which is
. Although those representations are
coherence
coherence meaning-changed coherence
A
comprehensive,
clarity
coherence clarity clarity
R
. Such implementation hinders the decoder
, such practice keeps the decoders
clarity
clarity clarity fluency
R
, local source information
source information generated by other encoder layers
clarity
clarity clarity clarity
R
explore to reuse the representations from different encoder layers
propose to encourage the decoder to take the full advantage of the multi-level source representations
clarity
clarity meaning-changed clarity
R
, that is,
. Concretely,
coherence
coherence coherence coherence
R
. We investigate multiple , representative strategies for cross-view coding, of which
and multiple strategies are explored to route the source representations. In particular,
clarity
clarity meaning-changed clarity
A
the
fluency
fluency coherence fluency
R
. Especially, GCA surpasses
, surpassing
clarity
coherence clarity clarity
R
machine translation
benchmark
clarity
clarity clarity clarity