Try reducing per_device_train_batch_size.
If you don’t want to reduce it drastically, try reducing max_seq_length from 128 to a lower number if you think your sequences are not that long enough to fit 128 token space.
Use a smaller model like Albert v2. In your use case, maybe distilBERT would be decent. That would take less RAM to perform forward pass.