What was the training and validation accuracy when run on the cpu?
Also, print out the loss in both the cpu and gpu . A different behavior in the cpu and gpu is actually very interesting given that most operations are the same.