I have a snippet of code that uses gradient checkpoints from torch.utils.checkpoint to reduce GPU memory:
    if use_checkpointing:
        res2, res3, res4, res5 = checkpoint.checkpoint(self.resnet_backbone, data['data'])
        fpn_p2, fpn_p3, fpn_p4, fpn_p5, fpn_p6 = checkpoint.checkpoint(self.fpn, res2, res3, res4, res5)
    else:
        res2, res3, res4, res5 = self.resnet_backbone(data['data'])
        fpn_p2, fpn_p3, fpn_p4, fpn_p5, fpn_p6 = self.fpn(res2, res3, res4, res5)

When I use gradient checkpointing, after training my model, the performance is worse. Does anyone know why this might be?