Hey @jogonba2, have you tried to verify the implementation of the copy mechanism? For example by using only the copy distribution (force setting p_gen to 0) and training and testing the model on the simple task of just copying the complete input to the output?
I’m currently trying to add the copy mechanism to T5 and currently my model is not able to do this yet.