If you want to create a trainable parameter or a tensor, which should be registered in the module during its initialization, you could use an nn.Parameter or register the tensor via self.register_buffer in the __init__ method.
This will make sure to transfer the tensors to the appropriate device when model.to() is called.
However, if you want to create a new tensor in the forward method (which is different than the original question), you could reuse the .device attribute of a known parameter or the input:
def forward(self, x):
    my_new_tensor = torch.randn(1, device=x.device)
    x = x + my_new_tensor
    return x