@tom Hi Thomas, thanks for your reply! I have a follow up question, what if I have:
a = torch.randn((4, 7), requires_grad=True)
c = torch.zeros(4, 7, requires_grad=True)
c[1, 1:3] = a[1, 1:3]
c[2, 3:4] = a[2, 3:4]
# instead of operating on a, we use c for future computation

And imagine a is calculated by some previous layers. I guess some values in a won’t receive gradient in backward pass. But can this save some gpu memory for me or not at all (compared to using a * mask for future computation)?
Best
Yujian