Can’t really help on this, have you checked the tutorial implementation to compare with your code ?

  
      
      GitHub
  
  
    

pytorch/examples 15

A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc.