Hey there, try replacing your forward with the following:
    def forward(self, z, y, run_on_ngpu=None):
        run_on_ngpu = run_on_ngpu or self.ngpu
        if isinstance(z.data, torch.cuda.FloatTensor) and run_on_ngpu > 1:
            output = nn.parallel.data_parallel(
                self, (z, y, 1), range(self.ngpu))
        elif run_on_ngpu == 1:
            output = self.main(z, y)
        else:
            raise RuntimeError
        return output

Happy PyTorching!