If the OOM error is raised before the forward pass is executed, this would mean that either your model or the current batch is already too large for your GPU.
Are you pushing the complete dataset to the device?