If I were in your position, I would try again after rm -rf t5-small
then verify in a python repl that
AutoConfig.from_pretrained('t5-small') doesn’t work
Then make a reproducible github issue, including transformers-cli env output.
This is not a distillation issue, and I can’t reproduce it on master.