It depends on the layers you are using in your model as well as e.g. if you set cudnn to use deterministic algorithms as described here 19.
If the absolute difference is in the range ~1e-6, it’ll be most likely due to the limited floating point precision.