Meet the same problem and solve it with https://github.com/fh295/semanticCNN 11
Specifically, to make the val dataset have the same structure as train set, you should run the following commands. It will move all validation pictures to corresponding class subfolders.
cd val/
wget -qO- https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh | bash