Solved by ptrblck in post #2 
              
              
                Your code basically looks alright. 
You could try the following to make the model learn: 

lower your learning rate and see if the loss is decreasing (a learning rate of 1e-3 might be a good starter for Adam
try to normalize your input. Currently it looks like you are dealing with some kind of categ…