Hi,
This is what buffers are for.
You can do self.register_buffer("foo", your_tensor). You can then use it as if you did self.foo = your_tensor.
And it will be moved around for you and saved into the state dict as well.