Can deepspeed and gradient checkpointing be used at the same time?
I'm finetuning roberta-large for some downstream tasks, and i'm wondering if deepspeed and gradient checkpointing can be used at the same for much more GPU memory reduction? Is there any conflict between these two?
Yes.

See the guide: https://huggingface.co/transformers/master/performance.html