What suraj said is correct.  We haven’t really tried this. What’s happening is you are calling the teacher decoder on the student encoder outputs.
Two options for you:
(1) If you pass --no_teacher (or just finetune.py your student), things should work.
(2) Otherwise (if you want to supervisee student logits ~= teacher logits) you need to roll up your sleeves and edit SummarizationDistiller.step to run the teacher encoder and student encoder.
I would recommend (1). (2) will be much slower and I suspect not much better performing.
Share your results!