Hi,
I have a model that has multiple outputs (20 outputs), and each of them is a linear classifier of 4-10 classes. In order to enable the model returns these 20 outputs could be:
# for example...
def Model(nn.Modules): 
    def __init__(self):
        ...
        self.features = nn.Sequential(...)
        # repeat the code N times
        self.classifier_1 = nn.Linear(1000, 7)
        self.classifier_2 = nn.Linear(1000, 4)
        ...
        self.classifier_N = nn.Linear(1000, 10)

    def forward(self, x):
        x = self.features(x)
        # repeat the code N times
        y1 = self.classifier_1(x)
        y2 = self.classifier_2(x)
        ...
        yN = self.classifier_N(x)
        return y1, y2, ..., yN

This approach works with single/multiple gpus.
However, repeating the classifier code self.classifier_X N times is not elegant neither dynamic/flexible. Here a new solution:
def Model(nn.Modules): 
    def __init__(self):
        ...
        self.features = nn.Sequential(...)
        self.nClassesPerClassifier = [7, 4, ..., 10]
        # create N classifiers
        for c in range(len(self.nClassesPerClassifier)):
            self.__setattr__('class_%d' % c,
                                nn.Linear(1000, nClassesPerClassifier[c]))

    def forward(self, x):
        x = self.features(x)
        # returns a dictionary whose key indicates the classN
        return {'class_%d' % c: self.__getattr__('class_%d' % c)(x) 
            for c in range(len(self.nClassesPerClassifier))}

This approach works for single GPU, but I got an error by using multiple GPU:
*** RuntimeError: maximum recursion depth exceeded while calling a Python object
The code of above works if returns a list or tuple of N Variables.
Does anybody know other solution to handle with multiple output with multiple gpus?