So the loss is the same for the first couple of epochs, but the gradients differ?
Are you seeing the difference in gradients from the beginning (which is strange, as the loss stays equal for some iterations) or after a while?