Hi,
I am working on something similar to L2L GDGD. I have a very small network which is applied element-wise over a tensor, the results are stacked and everything is returned.
Because this can be parallelised, I tried to spawn Processes using torch.multiprocessing as a first attempt. However I have noticed that transmitting the result tensors via a queue or pipe wipes out the grad_fn attribute of the tensor. But, if I were to pass a tensor in via Process(args=()) then grad_fn is preserved. A minimal working gist of a slightly different problem that shows the issue is attached below.
If this behaviour is intended, what are my options for speeding my model up? I only have a single GPU, and the network itself is around 120 params anyway, so I don’t think DataParallel can help all that much.

  
      gist.github.com
  
  
    https://gist.github.com/JosephDavidsonKSWH/3fe00f7319af9ae87df03c14019081b4 4
minimal_pytorch.py
import torch
import torch.multiprocessing as mp
from torch.autograd import Variable
import numpy as np


class ModProcess(torch.nn.Module):
    def __init__(self, out_size):
        super().__init__()
        self.out_size = out_size
This file has been truncated. show original