Using
x = torch.randn((self.X.shape), requires_grad=True)

does indeed work. It must be something with self.X sharing the same memory, with pself, so when I use pself.forward, it overwrites the old self.X, so that the new one does not have a gradient…