I am really grateful for your answer.
After I read some of your provided materials, I find that it seems using checkpoint to trade compute for memory only applies to ReLU functions, am I right?
Looking forward to your reply!