tobigue:

nly the copy distribution (force setting p_gen to 0) and training and testing the model on the simple task of just copying the complete input to the output?


Hi @tobigue.
I tested the p_gen=0 and p_gen=1 cases and the final distribution is the copy or the generation distribution respectively as expected. But I don’t tested it on “fully extractive” tasks.
Also, I did few experiments on my downstream task (keyword extraction) fixing the p_gen to the percentage of novel words and it seems to work better than learning the p_gen value. For some reason p_gen is almost always very close to 1, but I’m not sure it is a problem.
I think the implementation could be very similar for T5 models.