If you GPU is raising an OOM issue in the first training iteration using a batch size of 1, you could try to apply torch.utils.checkpoint to trade compute for memory or reduce the model or input length (if possible).
I cannot see any obvious errors in your code.
Note that you can add code snippets by wrapping them into three backticks ```, which makes debugging easier.