I don’t think the barrier causes the OOM, but might catch the error if it’s synchronizing the code.
To avoid it, you would have to lower the batch size or slim down the model or inputs.
Also, make sure that the GPU memory is not increasing in each iteration, as this might happen if you are storing the complete computation graph via storing tensors which are still attached to the computation graph.