Hi, possibly is a bit late but I was working on implementing the copy mechanism to MBart and released a gist: https://gist.github.com/jogonba2/ff9233023a406a45c655bbe090e3b05b 22
I never get better results using the copy mechanism. Most of the times, it works slightly better to use only the pretrained model without the copy mechanism. I’m trying to further pretraining MBartHez along with the copy mechanism to see what happens. Also, there are some weird things:

In my experiments, the p_gen is almost always between 0.97 and 0.99., so, the final distribution (copy+gen) is very similar to the distribution of the decoder (gen), even in extractive tasks.
During inference, the generate method gives a different output than trainer.predict.

The background for the implementation is this paper: https://aclanthology.org/2020.acl-main.125.pdf 3 . There is more information in the comments of the code.
Hope it helps!