So the best practice depends on the situation:

if they’re created once and then kept around and possibly updated, use buffers (m.register_buffer(…)). The optimizers won’t care about them, but they’ll be part of the state dict and loaded and saved etc. They will also be moved/cast along with the model in m.to. This is, for example, where batch norm tracks its parameters.
if you create newly while processing input, it is customary to either use the input or “any parameter” (next(n.parameters())) to get the device (and dtype).
There actually is another way: Tensors have .new_....(...) methods that some people like to use to get the device automatically. I must admit that to my mind this isn’t worth it and it overemphasizes the connection between the existing and the new tensor, so I would recommend to use the device= (and dtype=) keyword arguments. It also seems a bit more in the spirit of “there should be preferably only one way to do it” in the Zen of Python). Note that .new itself is something different and officially deprecated, but .new_... is considered OK by everyone but me.

Best regards
Thomas