Thank you, but I dont think your answer is relevant.
.clone() should not has any effect on this issue.
I also checked the situation that S2 = x.view(x.size(0), 512), the results are still the same and loss does not decreases.
I also tried diffeent lr : 0.1 and  0.0001. still no changes