Hi Rishav,
but does the loss go up or down or stay the same?
If you set the lr to 0, you should keep the same weights, so I’m tempted to say that it would be good to find where it stops to do funny things.
What optimizer are you using and which parameters? Sometimes when you do funny things with momentum or somesuch, it’ll get the strangest results.
Best regards
Thomas