@jphoward just so that i can reproduce this:

is this with imagenet? if not, how big is your dataset?
what GPU were you using?
what CUDA version did you install? did you use conda install pytorch torchvision cuda80 -c soumith ?