Here’s another torch.multiprocessing situation I can’t wrap my head around. This example is working:
import torch
import torch.multiprocessing as mp

def put_in_q():
    x = torch.IntTensor(2, 2).fill_(22)
    x = x.cuda()
    print(x)

p = mp.Process(target=put_in_q, args=())
p.start()
p.join()

But not this one:
import torch
import multiprocessing as mp

class CudaProcess(mp.Process):
    def __init__(self):
        mp.Process.__init__(self)

    def run(self):
        x = torch.IntTensor(2, 2).fill_(22)
        x = x.cuda()
        print(x)

p = CudaProcess()
p.start()
p.join()

The error I’m getting is:
terminate called after throwing an instance of 'THException'
  what():  cuda runtime error (3) : initialization error at .../pytorch/torch/lib/THC/THCGeneral.c:70