For any Tensor in the nn.Module, you can store it in self by doing self.foo = your_tensor or by doing self.register_buffer("foo", your_tensor).
I think that both will have the behavior that you want: you can access them by using self.foo and they can have history.
I think that the first one might be simpler to read as it is basic python semantic.
I can’t think of any reason why you need it to actually be a buffer (maybe I’m missing something though).