If you have multiple GPUs, you could use e.g. DistributedDataParallel to chunk the batch so that each model (and device) will process a smaller batch size.
Alternatively, you could lower your batch size or use torch.utils.checkpoint to trade compute for memory.