The issue for nn.DataParallel is that the model uses a hardcoded device as cuda:0, if CUDA is available (and cuda:1, which will fail so I assume 'cpu' should be used in the else statement).
However, nn.DataParallel will copy the model to each specified device in each forward pass, will split the input in dim0, and send each chunk to the corresponding device.
For this to work you cannot use hardcoded device arguments inside the model or forward, since nn.DataParallel will take care of it. If you need to create new tensors inside the forward, you would have to use the currently used device for this model replica (e.g. via new_tensor = torch.randn(1, device=x.device)).
While this code works fine on the GPU:
model = GenericNetwork(1., (1,), 1, 1, 1)
x = T.randn(8, 1)
out = model(x)
print(out)

You’ll get a device mismatch error, if you use model = nn.DataParallel(model):
RuntimeError: Expected tensor for 'out' to have the same device as tensor for argument #2 'mat1'; but device 1 does not equal 0 (while checking arguments for addmm)

for the aforementioned reason.