  Copy mechanisms explicitly obtain unchanged tokens from the source (input)
sequence to generate the target (output) sequence under the neural seq2seq
framework. However, most of the existing copy mechanisms only consider single
word copying from the source sentences, which results in losing essential
tokens while copying long spans. In this work, we propose a plug-and-play
architecture, namely BioCopy, to alleviate the problem aforementioned.
Specifically, in the training stage, we construct a BIO tag for each token and
train the original model with BIO tags jointly. In the inference stage, the
model will firstly predict the BIO tag at each time step, then conduct
different mask strategies based on the predicted BIO label to diminish the
scope of the probability distributions over the vocabulary list. Experimental
results on two separate generative tasks show that they all outperform the
baseline models by adding our BioCopy to the original model structure.

Computation and Language;Artificial Intelligence;Machine Learning