Solved by ptrblck in post #2 
              
              
                The last squeeze() operation is most likely not needed (x = x.squeeze(1)), but might be alright if your target has the same shape ([32]). 
Could you try to overfit a small data samples, e.g. just 10 samples and see if your model is able to do so by playing around with some hyperparameters. 
I think …