Hey there!
I am using your distillation script 9 (thanks for sharing it!) and based on the dumped checkpoints I see, it seems that they contain both the teacher and the student.
Assuming that my observation is correct, how can I dump only the student sub-model?
@sshleifer wondering if you have any thoughts.