Thanks a ton ptrblck this makes a lot of sense. Throwing a half a million images at it is probably the problem here. I’ve removed norm, and been using half so I’ll just try with AlexNet for now, and use a small subset of the full dataset.