Solved by ptrblck in post #9 
              
              
                Thanks for the executable code, that was really helpful. 
You are accidentally broadcasting the loss, since you have a mismatch in the output and target tensors. 
While your output has the shape [batch_size, 1], the target has [batch_size]. 
This yields to a broadcasting as seen here: 
# your code w…