The following multiprocessing code works fine with DEVICE=cpu, but seems to fail with CUDA:
import torch
import torch.multiprocessing as mp

DEVICE = "cuda"


def proc(queue):
    queue.put(torch.tensor([1.], device=DEVICE))


if __name__ == "__main__":
    ctx = mp.get_context("spawn")
    queue = ctx.Manager().Queue()
    p = ctx.Process(target=proc, args=(queue,))
    p.daemon = True
    p.start()
    print("received: ", queue.get())

throws an error:
Traceback (most recent call last):
  File "examples/ex2.py", line 17, in <module>
    print("received: ", queue.get())
  File "<string>", line 2, in get
  File "/home/npradhan/miniconda3/envs/pytorch-master/lib/python3.6/multiprocessing/managers.py", line 772, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError:
---------------------------------------------------------------------------
Unserializable message: Traceback (most recent call last):
  File "/home/npradhan/miniconda3/envs/pytorch-master/lib/python3.6/multiprocessing/managers.py", line 283, in serve_client
    send(msg)
  File "/home/npradhan/miniconda3/envs/pytorch-master/lib/python3.6/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/home/npradhan/miniconda3/envs/pytorch-master/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
  File "/home/npradhan/miniconda3/envs/pytorch-master/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 161, in reduce_tensor
    (device, handle, storage_size, storage_offset) = storage._share_cuda_()
RuntimeError: invalid device pointer: 0x7f89d4200000 at /home/npradhan/workspace/pyro_dev/pytorch/aten/src/THC/THCCachingAllocator.cpp:262

---------------------------------------------------------------------------

I also noticed that the error thrown is different across slightly different variants of the above code. e.g. using mp.Queue instead of mp.Manager().Queue() seems to throw a cuda runtime error (30). Changing to using a FloatTensor constructor instead of a tensor constructor throws RuntimeError: legacy constructor for device type: cpu was passed device type: cuda, but device type must be: cpu. I think my mental model might be incorrect - so any help understanding what is happening here would be appreciated.