Hi @patrickvonplaten  and @sgugger , I am facing the same issue, constrast_loss is not changing after 3 backward it goes straight to zero and didn’t change even after many epochs, any idea how can we reproduce the results. Thanks