I was testing a basic neural network one hidden layer. I tried various loss functions in the same session and the net got stuck in what I assume was a local min but it was completely wrong. I restarted the session and  1 epoche was better than 1000. I am using ADAM. My belief is that changing the error function drastically  caused this mistake.  I was wondering if anyone has encountered this problem in the past and how they dealt with it; and the command used to reset weights within a session.