It sounds to me like if your model has some Dropout layers, which will explain the behavior you’re experiencing. At test time, dropout doesn’t do anything, but during training it randomly drops some units. Take a look at the documentation for Dropout 3.