Not at my computer, but loosely describing, you need to use register_buffer() to create this variable, then it will be magically copied between gpus. Only registered buffers and nn.Parameters enjoy this behavior.
Hope this helps.