Thank you for your suggestion! I changed according to what you suggest,but the validation accuracies stayed the same. At least now I could confirm my autograd is working, and I will look deeper to the network design.