That sounds quite strange, as I would assume the same loss is generated using the same data and model parameters. So the gradients / weight updates seem to differ somehow?
Could you post a code snippet reproducing this behavior?